Lecture Notes in 
Computer Science 1790 



Nancy Lynch Bruce H.Krogh (Eds.) 



Hybrid Systems: 
Computation 
and Control 

Third International Workshop, HSCC 2000 
Pittsburgh, PA, USA, March 2000 
Proceedings 




Springer 




Lecture Notes in Computer Science 1790 

Edited by G. Goos, J. Hartmanis and J. van Leeuwen 




springer 

Berlin 

Heidelberg 

New York 

Barcelona 

Hong Kong 

London 

Milan 

Paris 

Singapore 

Tokyo 




Nancy Lynch Bruce H. Krogh (Eds.) 



Hybrid Systems: 
Computation 
and Control 



Third International Workshop, HSCC 2000 
Pittsburgh, PA, USA, Mareh 23-25, 2000 
Proeeedings 




Springer 




Series Editors 



Gerhard Goos, Karlsruhe University, Germany 
Juris Hartmanis, Cornell University, NY, USA 
Jan van Leeuwen, Utrecht University, The Netherlands 

Volume Editors 
Nancy Lynch 

Massachusetts Institute of Technology 
Laboratory for Computer Science 
Cambridge, MA 02139, USA 
E-mail: lynch@theory.lcs.mit.edu 

Bruce H. Krogh 
Carnegie Mellon University 

Department of Electrical and Computer Engineering 
Pittsburgh, PA 15235, USA 
E-mail: krogh@ece.cmu.edu 

Cataloging-in-Publication Data applied for 

Die Deutsche Bibliothek - CIP-Einheitsaufnahme 

Hybrid systems : computation and control ; third international 
workshop ; proceedings / HSCC 2000, Pittsburgh, PA, USA, March, 23 - 
25, 2000. Nancy Lynch ; Bruce H. Krogh (ed.). - Berlin ; Heidelberg ; 
New York ; Barcelona ; Hong Kong ; London ; Milan ; Paris ; Singapore ; 
Tokyo : Springer, 2000 

(Lecture notes in computer science ; Vol. 1790) 

ISBN 3-540-67259-1 



CR Subject Classification (1991): Cl.m, F.3, C.3, D.2.1, F.1.2, J.2, 1.6.1 
ISSN 0302-9743 

ISBN 3-540-67259-1 Springer- Verlag Berlin Heidelberg New York 



This work is subject to copyright. All rights are reserved, whether the whole or part of the material is 
concerned, specifically the rights of translation, reprinting, re-use of illustrations, recitation, broadcasting, 
reproduction on microfilms or in any other way, and storage in data banks. Duplication of this publication 
or parts thereof is permitted only under the provisions of the German Copyright Law of September 9, 1965, 
in its current version, and permission for use must always be obtained from Springer- Verlag. Violations are 
liable for prosecution under the German Copyright Law. 

Springer- Verlag is a company in the BertelsmannSpringer publishing group. 

© Springer-Verlag Berlin Heidelberg 2000 
Printed in Germany 

Typesetting: Camera-ready by author, data conversion by Firma Steingraber 
Printed on acid-free paper SPIN 10720042 06/3142 5 4 3 2 1 0 




Preface 



This volume contains the proceedings of the Third International Workshop on 
Hybrid Systems: Computation and Control (HSCC 2000), which was held on 
March 23-25, 2000, in Pittsburgh, Pennsylvania. The proceedings of the first 
two workshops in this series were published by Springer-Verlag, in the Lecture 
Notes in Computer Science series, as volumes 1386 and 1569. 

The focus of the Hybrid Systems workshop series is on modeling, control, 
synthesis, design, and verification of hybrid systems. A hybrid system is a theo- 
retical model for a computer controlled engineering system, with a dynamics that 
evolves both in a discrete state set and in a family of continuous state spaces. Hy- 
brid systems research is motivated by, for example, control of electro-mechanical 
systems (robots), air traffic control, control of automated freeways, and chemical 
process control. The research area of hybrid systems overlaps both with com- 
puter science and with control theory. The workshop series is intended to foster 
the interaction between researchers from these fields in addressing problems in 
this new domain. 

The scientific program of the workshop consisted of four invited talks and 
32 contributed talks. The following researchers presented invited talks: K. Butts 
(Ford Research, USA), N. Leveson (MIT, USA), A. Sangiovanni-Vincentelli (U. 
California, Berkeley, USA), and B. Williams (MIT, USA). The contributed talks 
were based on the papers in these proceedings. 

The program committee, chaired by the editors, selected the 32 contributed 
papers out of 71 submitted papers. The editors are grateful to the members of 
the program committee for their generous help in the reviewing and the selection 
process. 

The editors are grateful to the speakers and all the other workshop partic- 
ipants, and to the sponsoring institutions whose support has made this event 
possible. Finally, they would like to thank George Woodzell for his system sup- 
port, Drew Danielson for his help with local arrangements, and Joanne Talbot 
for all her hard work in assembling this proceedings volume. 
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Hybrid Models for Automotive 
Powertrain Systems: Revisiting a Vision 



Ken Butts 



Powertrain Control Systems 
Ford Research Laboratory 
Dearborn, MI 
kbutt sl@ford.com 



Abstract. Due to the persistent need to develop increasingly complex 
systems with improved quality and reduced development effort, automo- 
tive manufacturers are employing model-based development approaches 
wherever sensible. This is particularly true for powertrain control system 
development, as domain relevant computer-aided control system design 
tools have become commercially available. It is now possible to model and 
simulate the powertrain system dynamics in closed-loop with detailed be- 
havioral models of the control algorithm. These control algorithm models 
capture nominal, initialization, diagnostic, and failure-mode-effects man- 
agement modes of operation to the extent that simulation-based valida- 
tion and verification procedures can be employed. These procedures help 
to ensure that the algorithm design and its associated software realiza- 
tion meet the system requirements with quality. 

Simulation-based development (design, validation, and verification) 
methods only evaluate the system’s behavior under the initial condi- 
tions, input scenarios, and parameter values as defined in the simula- 
tion test-suite. Thus, comprehensive validation and verification is expen- 
sive and time consuming. (Of course, exhaustive system validation and 
verification is impossible in a simulation-based development approach.) 
Importantly, given that powertrain models are being created to support 
mainline development, we now have an opportunity to go beyond simula- 
tion by employing systems analysis methodologies in the design process. 
The purpose of this talk is to describe hybrid systems analysis queries 
that, if answered in an efficient and intuitive way, would be a boon to 
the powertrain controller development community. 

We begin by stating two analysis tool objectives that are derived from 
our experiences in using these models in a production development envi- 
ronment. First, wherever possible, we desire that our analysis methods 
be based on commercial tools. Second, we desire to analyze the models 
in the styles (one for physical plant models and another for control al- 
gorithm models) that they are currently built. We desire to analyze the 
models in the accepted styles because the model preparation requires sig- 
nificant engineering effort and it is unlikely that the organization could 
support the additional expertise, training, and effort required to specially 
prepare alternative analysis models. 

Next we describe the modeling styles that are used in the production 
development process and provide an example based on an automatic 
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transmission control system. The physical plant model is comprised of a 
two-state engine model, a quasi-static torque converter model, the trans- 
mission dynamics for first gear, second gear and the one-to-two gearshift, 
and a simple longitudinal vehicle dynamics model. The discussion focuses 
on the hybrid nature of several components within the system. The as- 
sociated control algorithm model is comprised of an abstract transmis- 
sion shift scheduler and simplified shift control logic for the one-to-two 
gearshift and the one-to-two-to-one “change-of-mind” gearshift. 

We list analysis queries that would enhance the powertrain controller 
development process if they were available. We also discuss the specific 
application of these queries to the transmission control example. These 
analyses include stability within a mode of operation, modal transition 
integrity, safety, and liveness. 

We conclude with the remark that we have a wealth of new information 
available in the automotive powertrain controller development process: 
formal and detailed models of the system’s behavior. We hope to be able 
to fully exploit these models through analysis. 




Experiences in Designing and Using Formal 
Specification Languages for Embedded Control 

Software 



Nancy G. Leveson 

Aeronautics and Astronautics Department, 
Massachusetts Institute of Technology 
Cambridge, MA USA 
levesonSmit . edu 



Abstract. For the past ten years, I have been designing formal specihca- 
tion languages for specifying software requirements on complex systems. 
In order to understand what is needed in such languages, my students 
and I have been applying our ideas to real systems and using what we 
have learned to generate new hypotheses about what is needed to make 
such languages both useful and used. This research is part of a larger 
effort to assist in developing safety-critical embedded systems. 

Some of the lessons we have learned: 

1. Formal specifications can be practical in industry, but the notations 
need to be readable and reviewable by those who will be using them, 
not just by Ph.D. computer scientists. Most specification errors will 
be found by domain experts reading the specification, not by formal 
analysis tools (although tools can be useful, particularly in helping 
designers understand the specifications). 

2. The problems involved in specifying large, complex systems are dif- 
ferent than the problems involved in specifying the simple examples 
usually found in research papers. If we want our languages to be 
used, we need to start from real problems from the beginning and 
not simply eliminate all the parts of the problem we cannot handle. 

3. Some common features of formal specification languages are very 
error-prone in use and should be eliminated from our languages. 

4. Our languages must support building complex models. Support in- 
cludes tools to assist in writing, visualizing, and validating such spec- 
ifications. 

5. Formal models and specifications are very expensive to produce. 
They will not be adopted by industry unless the payoff is worth 
it. To date, that has not been true. They will be used if we can solve 
problems with them that they cannot solve adequately in simpler or 
cheaper ways or that are important enough to them to be worth the 
investment. 

Our current research goals include: integrating formal and informal spec- 
ifications, adding ’’intent,” supporting human problem solving (using 
what is known about this by cognitive psychologists), providing more 
assistance in building formal specifications, and devising analysis tools 
and algorithms to assist with important problems found in industrial 
projects. The talk will provide more details and examples. 
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for Robotic Space Exploration 
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Abstract. A new generation of sensor rich, massively distributed sys- 
tems is emerging that offers the potential for profound economic and 
environmental impact, including building energy systems, deep space 
probes and sensor webs that monitor the earth ecosystem. These robotic 
webs have the richness that comes from interacting with physical envi- 
ronments, together with the complexity of networked software systems. 
They must be efficient, capable and long lived, that is, able to survive 
decades of autonomous operation within unforgiving environments. 
Model-based autonomy meets this challenge through two ideas. First, we 
note that programmers generate the desired function based on their com- 
monsense knowledge of how the software and hardware modules behave. 
The idea of model-based programming is to exploit this modularity by 
having engineers program reactive systems by simply articulating and 
plugging together these commonsense models. The second challenge is 
the infeasibility of synthesizing a set of codes at compile time that envi- 
sion all likely failure situations and responses. Our solution is to develop 
real time systems, called model-based executives that respond to novel 
situations on the order of hundreds of milliseconds, while performing ex- 
tensive deduction, diagnosis and planning within their reactive control 
loop. 

In this talk I will formulate a model-based executive as a deductive form 
of an optimal, model-based controller, in which models are specified 
through a combination of concurrent, probabilistic transition systems 
and propositional logic. This framework allows us to unify a diverse set 
of research results from model-based reasoning, planning, search, real- 
time propositional inference, and the theory of reactive languages. I will 
then discuss how reactivity is achieved using a high performance deduc- 
tive kernel, called OPSAT that solves combinatorial optimization prob- 
lems with constraints encoded in propositional logic. A first generation 
executive, called Livingstone, was demonstrated this year on NASA’s 
first autonomous space probe, called Deep Space One, shortly before 
its asteroid encounter. Livingstone is also being demonstrated in a vari- 
ety of space systems that include Mars rovers, Martian chemical plants, 
multi-spacecraft telescopes and the next generation shuttle. Finally, I 
will touch on future research that shifts from controlling the internals of 
single robotic systems to webs of robotic vehicles. 
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Models of Computation and Simulation 
of Hybrid Systems 



Alberto Sangiovanni-Vincentelli 



The Edgar L. and Harold H. Buttner Chair 
of Electrical Engineering and Computer Science, 
Department of EECS 
University of California at Berkeley 



Abstract. A design (at all levels of the abstraction hierarchy from func- 
tional specification to final implementation) is generally represented as a 
set of components, which can be considered as isolated monolithic blocks, 
which interact with each other and with an environment that is not part 
of the design. The model of computation defines the behavior and in- 
teraction of these blocks. Compactness of description, fidelity to design 
styles, ability to simulate, synthesize to an appropriate implementation 
and optimize its behavior are criteria to follow for the choice of an MOC 
to describe and manipulate a design. For example, some MOCs are suit- 
able for describing complicated data transfer functions and completely 
unsuitable for complex control, while others are designed with complex 
control in mind. 

We review the foundations of a theory of models of computation (MOC) 
(see Lee and Sangiovanni-Vincentelli, IEEE Trans. CAD, Dec. 1998). We 
will try to convey the basic notions and definitions to avoid ambiguity 
that often arises when MOCs are used in a non-rigorous fashion. We also 
believe that some degree of confusion has arisen in the hybrid system 
community due to an improper use of MOCs. 

Hybrid systems in the general sense of the term could be considered as 
formalisms used to describe a complex system as combinations of MOCs 
where a single one is not powerful or expressive enough. When a hybrid 
system is simulated, the MOCs used to describe its behavior dictate the 
way the components of the system interact and execute. Since MOCs 
differ mostly for the way their components interact, the most difficult 
problem to solve when simulating them is to resolve the interfacing issue. 
We will review the issues and the ways used to cope with them. We 
will draw from the large bag of tricks developed over the years in the 
simulation community (especially for circuit simulation, e.g. SPICE, that 
exhibits some of the problems faced by the hybrid system community) 
to document difficulties and successes. 
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Modular Specification of Hybrid Systems in 

Charon 



Rajeev Alur, Radu Grosu, Yerang Hur, Vijay Kumar, and Insup Lee 

Department of Computer and Information Science, University of Pennsylvania, 
Philadelphia PA 19104-6389, USA, 

{alur , grosu, yehur,kumar, lee }@cis .upenn. edu, 
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Abstract We propose a language, called Charon, for modular spec- 
ification of interacting hybrid systems. For hierarchical description of 
the system architecture, Charon supports building complex agents via 
the operations of instantiation, hiding, and parallel composition. For hi- 
erarchical description of the behavior of atomic components, Charon 
supports building complex modes via the operations of instantiation, 
scoping, and encapsulation. Features such as weak preemption, history 
retention, and externally defined Java functions, facilitate the description 
of complex discrete behavior. Continuous behavior can be specified using 
differential as well as algebraic constraints, and invariants restricting the 
flow spaces, all of which can be declared at various levels of the hierar- 
chy. The modular structure of the language is not merely syntactic, but 
can be exploited during analysis. We illustrate this aspect by presenting 
a scheme for modular simulation in which each mode can be compiled 
solely based on the locally declared information to execute its discrete 
and continuous updates, and furthermore, submodes can integrate at a 
finer time scale than the enclosing modes. 



1 Introduction 

A hybrid system typically consists of a collection of digital programs that interact 
with each other and with an analog environment. The design and implementa- 
tion of hybrid systems remains a challenging task. We believe that availability 
of a specialized design language for hybrid systems will aid the developers sig- 
nificantly and lead to opportunities for greater design automation. Traditional 
tools for modeling and simulation of dynamical systems, such as Matlab (see 
http://www.mathworks.com), provide little support for modular specifications. 
On the other hand, modern software design languages, such as Statecharts [10] 
and Uml [6], provide no support for describing continuous behavior. In this pa- 
per, we introduce a language, called Charon, for hierarchical specification of 
interacting hybrid systems. The design of our language was guided by two con- 
cerns. First, the language should support state-of-the-art modeling concepts such 
as encapsulation, reuse, preemption, and hierarchy. Second, it should be possi- 
ble to give a modular formal semantics to the language which can be exploited 
during simulation, verification, and code generation. 
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In Charon, a system is described as a collection of agents communicating 
via shared variables, and the behavior of each agent is specified by a hierarchical 
state machine. Key features of Charon are summarized below. 

Architectural hierarchy. The building block for describing the system ar- 
chitecture is an agent that communicates with its environment via shared 
variables. The language supports the operations of composition of agents to 
model concurrency, hiding of variables to restrict sharing of information, and 
instantiation of agents to support reuse. 

Behavior hierarchy. The building block for describing flow of control inside an 
atomic agent is a mode. A mode is basically a hierarchical state machine, that 
is, a mode can have submodes and transitions connecting them. Variables 
can be declared locally inside any mode with standard scoping rules for 
visibility. Modes can be connected to each other only via well-defined entry 
and exit points. We allow sharing of modes so that the same mode definition 
can be instantiated in multiple contexts. Finally, to support exceptions , the 
language allows group transitions from default exit points that are applicable 
to all enclosing modes, and to support history retention, the language allows 
default entry transitions that restore the local state within a mode from the 
most recent exit. 

Discrete updates. Discrete updates are specified by guarded actions label- 
ing transitions connecting the modes. We assume interleaving semantics for 
concurrency (i.e., only one atomic agent is executed in a discrete round), 
run-to- completion semantics for individual agents (i.e., once an agent is cho- 
sen for discrete update, it keeps executing its transitions as long as there are 
enabled ones), and higher priorities for inner modes (i.e., group transitions 
from the default exit of a mode are examined only when there are no enabled 
transitions inside). 

Continuous updates. Some of the variables in Charon can be declared ana- 
log, and they flow continuously during continuous updates that model pas- 
sage of time. The evolution of analog variables can be constrained in three 
ways: differential constraints (e.g. by equations such as i = f(x,u)), alge- 
braic constraints (e.g. by equations such as y = g{x, u)), and invariants (e.g. 
\x — y\ < e) which limit the allowed durations of flows. Such constraints can 
be declared at different levels of the mode hierarchy. 

It should be noted that Charon is a modeling language: it supports nonde- 
terminism for both discrete and continuous updates, it is suitable for describing 
the system as well as the assumptions about the environment in which the system 
is supposed to operate, and for describing the same system at different levels of 
abstraction. The language constructs primarily facilitate the description of con- 
trol flow, but it also supports calls to externally defined Java functions which 
can be used to write complex data manipulations. 

After introducing the language in the next two sections, we proceed to il- 
lustrate how to exploit the modular structure during simulation. Since modes 
are hierarchical, multiple modes within an atomic agent can be active simulta- 
neously, and a large number of transitions may be applicable in a given state. 
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In our modular scheme for discrete updates, each mode gets compiled into a 
function which gets control at one of its entry points along with an input global 
state, and returns the control at one of its exit points together with a modified 
global state. Such a modular scheme is possible since Charon modes have ex- 
plicit entry and exit points including the default ones, and inner transitions have 
higher priorities over the outer ones. 

Introducing modularity in simulation of time rounds is more challenging. 
Since time is global, update of analog variables of all agents must be synchro- 
nized. Furthermore, within a single agent multiple modes are active, and the 
constraints on continuous update may be defined at any level of the hierar- 
chy. This implies that simulating a flow requires solving constraints of all active 
modes of all agents simultaneously. In a modular scheme, we wish to compile 
each mode independently of the other. 

Concurrency. To handle concurrency, we propose a scheme for distributed sim- 
ulation in which each agent has its own local clock. The scheme ensures that 
the differences among local clocks are bounded. 

Hierarchy. Each mode is responsible for integrating the variables whose update 
laws are defined locally, at a time scale of its own choice based on the local 
control laws and the invariants. A mode M is invoked from higher level with 
an input state, a bound 5 on integration time, and an invariant constraint 
on the local variables of M. The integration within M assumes that the 
variables whose update laws are defined outside M stay unchanged. It can 
choose to integrate at time intervals shorter than 5, and can use integration 
routines of its submodes as black-boxes. 

In summary, instead of solving the entire set of constraints simultaneously, 
the modular scheme computes the approximate solutions by layering the con- 
straints as dictated by the modular specification. 

Related work. Early formal models for hybrid systems include phase transition 
systems [13] and hybrid automata [1]. There has been a lot of research concern- 
ing analysis of hybrid automata leading to the model checker HyTech [5,11]. 
Models such as hybrid I/O automata [12] and hybrid modules [4] allow compo- 
sitional treatment of concurrent hybrid behaviors. None of these models admit 
hierarchical specifications. 

The notion of hierarchical state machines was introduced in State- 
charts [10], and is present in many software design paradigms such as Uml [6]. 
Our treatment of hierarchy is closest to hierarchical reactive modules [2] which 
shows how to define a modular semantics for hierarchical (discrete) modes. 

The languages Shift [8] and HyCharts [9] allow hierarchical specifications 
of hybrid behavior, and Stateflow (see http://www.mathworks.com) allows 
hierarchical specifications of dynamic behavior. However, modular simulation 
has not been a concern in the design of these languages. Furthermore, Charon 
supports new features such as preemption and reuse that are important from a 
programming perspective. 
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2 Language Overview 

A hybrid system is described in Charon by a set of agents communicating over 
a set of shared variables in an asynchronous way. 

The agents may be grouped together in a hierarchical way into composed 
agents starting from the most primitive ones called atomic agents. Information 
flow inside a composed agent may be hidden to the outside world. The grouping 
of agents into composed agents gives the architecture of the hybrid system. A 
composed agent may also be understood as an architectural pattern that may 
be instantiated, i.e., reused in different contexts that match the pattern. 

For example, at a lower level, a robot may be understood as the composition 
of a sensing agent, a controller agent, and an actuator agent. At a higher level, 
one may consider a team of cooperating robots, communicating with each other 
in order to achieve a common goal. 

The behavior of an atomic agent is given by a set of modes that are linked 
together by a set of transitions. Each mode represents a particular behavior of the 
agent and has an associated dynamics given by a set of algebraic and differential 
constraints. The dynamics may be further constrained by a set of invariants. 
Modes may also be grouped together in a hierarchical way to form composed 
modes starting from the most primitive ones called leaf modes. Moreover, each 
mode may declare its own set of local variables that is hidden outside the mode, 
but is accessible to its submodes. 

In other words, a mode is a sequential, communicating, hierarchical state 
machine with well defined dynamics, interfaces, and scoping rules for variables 
similar to structured programming languages. It may be also regarded as a be- 
havioral pattern that may be instantiated. 

For example, at a lower level, one may consider for a robot the modes walk- 
Forward, walkLeft, walkRight and walkBackward. At a higher level one may 
consider the modes avoidObstacle and track Wall. 

Note that an atomic agent is nothing but a hierarchical mode. Its variables 
and behavior are completely determined by the mode. Moreover, a hierarchical 
agent is nothing but a set of hierarchical modes with local variables determined 
by the agent hierarchy. So why do we distinguish between modes and agents? The 
answer is that encapsulating modes inside agents prevents parallel composition 
inside modes, i.e., modes are entities composed in a purely hierarchical way. 

Refer to [3] for more details and examples. 

2.1 Variables 

Discrete and analog variables. A hybrid agent has a finite set of typed 
variables denoted A.V. Some of these variables are updated in a discrete fashion 
and the others change in an analog fashion when time elapses. Accordingly, the 
set A.V is partitioned in two sets, the set A.dscV of discrete variables and the 
set A.anaV of analog variables. 

Differential and algebraic variables. In control theory it is common to com- 
pute the values of the analog variables A.anaV by using algebraic and differential 
equations. For example, x=f{x,u) is a differential equation whereas y=g{x,u) 
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is an algebraic equation. Regarding / and g as functional blocks and x, y, u as 
wires, it is easy to see that the wire a; is a feedback loop of /. As a consequence, 
the current value of the output x oi f depends on the previous (infinitisimal) 
value of X. In contrast, the current value of the output y of g depends only on the 
current values of the inputs x and u. Hence, an algebraic equation is very similar 
to a combinational circuit whereas a differential equation is similar to a sequen- 
tial circuit. In Charon we generalize algebraic equations also to inequalities. We 
call the differential equations and algebraic equations generically as constraints. 
The variables defined by algebraic constraints are called algebraic variables and 
the variables defined by differential constraints are called differential variables. 
Hence, A.anaV = A.diffV U A.algV . We insist that A.diffV n A.algV = 0. Note 
that hybrid automata do not make any distinction between these two kinds of 
variables. 

Permitted read/write accesses. The variables A.V of an agent A are classi- 
fied according to their visibility and update permissions into three sets: the set 
A.lclV of local variables that cannot be read or written by other agents, the set 
A.wrtV of write variables that are written by A, and can be read by other agents, 
and the set A.readV of read variables that are read by A, and may be written by 
other agents. The sets A.readV and A.wrtV need not be disjoint. Similarly, the 
set of local variables A.lclV may be both read and written. The set of read and 
write variables A.gblV = A.readV U A.wrtV is used for communication and it is 
called the set of global variables. The set A.updV = A.wrtV U A.lclV of write and 
local variables is called the set of updated variables. Hence, our communication 
model is that of asynchronous communication over shared variables. This model 
is a very general and allows to define channels as a special case. 

States and actions. Given a set V of typed variables, a state over R is a 
function mapping variables to their values. Given two sets V and W of variables, 
an action from H to W is a binary relation between the states over V and the 
states over W . In Charon specifications, an action consists of an action guard 
over V and an action body from V to W. We say that an action is enabled 
(disabled) at a state s if its guard is true (false) at that state. 

2.2 Hierarchical Modes 

Hierarchy. A mode in Charon has a very refined control structure, given by 
a hierarchical, hybrid state machine. It basically consists of a set of submode 
references connected by transitions such that at each moment of time only one 
of the submode references is active. A submode reference has associated again a 
mode and we require that the modes form an acyclic graph with respect to this 
association. By using modes and mode references several references may share 
the same mode. This is highly desirable because modes in a definition are never 
simultaneously active. A mode resembles an or state in Statecharts, but it 
has more powerful structuring mechanisms. 

Variables. A mode has global as well as local variables. Global variables are 
used to share data with the environment of a mode , and are classified into 
the set ready of read variables and the set wrtV of write variables. The set 
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Fig. 1. Scoping rules and transition types 



gblV = ready UwrtV is called the set of global variables. The set of local variables 
IclV of a mode is accessible only by its transitions and submodes. Thus, the 
scoping rules for variables are as in standard structured programming languages. 
For example, in Figure 1 left, the transitions of the mode p (like r, s, and t) 
may refer only to the variables x, y and z. These variables are global to the 
modes referred to by m and n. However, the variables local in the mode referred 
to by m may not be used in the mode referred to by n. For example, in Figure 1 
left, the variable z may be accessed both in m and n but the variables u and v 
are private to m and n, respectively. 

Dynamics. A mode has an associated set of constraints. These include dif- 
ferential equations, algebraic equations and invariants that are differential and 
algebraic equations or inequalities. The constraints define the flows of the mode, 
i.e., the way analog variables are updated while the agent is in this mode. The 
invariants define conditions that have to be satisfied by the variables in this 
mode, i.e., they define allowed durations. The scoping rules also apply for these 
constraints. For example, in Figure 1, constr_p may only refer to x, y, and 
z and constrjn may refer only to z and u. For each differential and algebraic 
variable updated by a mode we require that the variable is either updated by 
the mode itself or it is updated by all submodes of this mode. For example, in 
Figure 1, the local variable z is either updated by a constraint in the mode p or 
by constraints in both submodes m and n. 

Interfaces. To obtain a modular language, we require the modes to have well 
defined control points classified into entry points (marked as white bullets) and 
exit points (marked as black bullets). The transitions connect the control points 
of a mode and its submode references to each other. For example, in Figure 1 
right, a is an entry transition, g, h, and j are exit transitions, b is an entry /exit 
transition, and c and i are internal transitions. Between these transitions there is 
a subtle difference. Entry transitions initialize the local variables by reading only 
from the global variables. Exit transitions forget the values of the local variables 
by writing only to the global variables. It is only the internal transitions that 
may both read and write the local variables. 

Preemption. To model preemption we use the special default exit point dx. 
A transition starting from the default exit point of a mode is called a group 
transition. It may be taken whenever the control is inside the mode and no 
internal transition is enabled. For example, in Figure 1 right, the group transition 
d is taken if it is enabled and all the transitions c , g, h, i , and j are disabled. 
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Hence, inner transitions have a higher priority than the group transitions, i.e., 
we use weak preemption (like the weak kill in Unix, versus the strong kill -9). 
This definition of priorities allows us to define in Section 4 a modular simulation. 
History. To allow history retention, we use the special default entry point de. 
A transition entering the default entry point of a mode restores the values of 
all local variables along with the position of the control (a transition may enter 
a default entry of a mode only if the mode was left along its default exit). For 
example, both transitions e and f in Figure 1 right, enter the default entry point. 
The transition e is called a self group transition. A self transition (like e) or 
more generally a self loop like d, q, and f may be understood as an interrupt 
handling routine. While a self loop may be arbitrary complex, a self transition 
may do simple things like counting the number of occurrences of an event (e.g., 
clock events). 

The set of modes in a Charon specification is supposed to be globally accessi- 
ble. Moreover, since a mode may refer to other modes we require that referencing 
forms an acyclic graph. 

Leaf and top level modes. A leaf mode is a mode with no submodes and a 
default identity transition from its default entry point de to its default exit point 
dx. A top-level mode is a mode M with a single explicit entry point e and no 
exit points. 

Mode operations. The mode definition can be viewed as an encapsulation 
operator over its submodes, and thus, modes are constructed from leaf-modes 
using encapsulation repeatedly in a non-recursive manner. 

2.3 Hierarchical Agents 

An atomic agent is basically a top level mode whose global variables are used 
for communication with other agents. As we already mentioned, atomic agents 
may be composed to form composed agents and communication inside composed 
agents may be hidden. Intuitively, composition of atomic agents is the union of 
their modes and hiding is a declaration of local variables. To make the operations 
over agents closed under composition and hiding, we define an agent as follows. 
Definition 1. (Agent) An agent P is a tuple consisting of 
Modes. A set of top-level modes M . 

Local variables. A set IclV C Um^M'm.gblV of local variables. 

Global variables. A set gblV = (UmGM'm.gblV) \ IclV of globals variables. 

Definition 2. (Composition) If A and B are two agents, then the composition 
A\\B is the agent with the set IclV = A.lclV U B.lclV of local variables, the 
set wrtV = A.wrtV U B.wrtV of write variables, the set readV = A.readV U 
B. ready of read variables and the set M = A.M U B.M of top level modes. 
Definition 3. (Variable Renaming) Let A he an agent, x G A.gblV a global 
variable of the agent and y ^ A.V a variable of the same type as x but not 
contained in A. Then the renaming A[x := y] is the agent obtained by consistently 
renaming x by y in A.V and in all modes m G A.M . 

Definition 4. (Variable Hiding) Let A he an agent, x G A.gblV a global variable 
of the agent. Then the variable hiding hide x in. A is the agent obtained by 
replacing A.gblV with A.gblV \ {a;} and A.lclV with A.lclV U {a;}. 
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3 Global Semantics 

One alternative in giving a semantics to a hierarchical system is to consider 
hierarchy as just a convenient syntactic abbreviation. This reduces the semantic 
definition to two considerably easier subproblems: a) show how to construct a 
fiat system out of the hierarchical one and b) give a semantics to the fiat system. 

3.1 The Flattening Operation 

Given a mode definition, the flattening operation recursively eliminates the sub- 
mode references as follows: a) take for each reference m the associated definition, 
b) prefix all elements of the mode definition by m, c) continue recursively until 
all references point to a leaf mode definition. The set of elements obtained this 
way are taken as the elements of the fiat mode. 

As a consequence of flattening, all elements of the resulting mode are pre- 
fixed with a path mi'.m2'- ■ ■ ■ 'rak from the root mode reference mi down to the 
containing mode reference mk of the original hierarchical mode. For example, 
a control point c has now the form ■ ■ ■ '.mk-c. The set of local variables 

flat{M).lclV of the flattened mode flat{M) is the transitive closure of the local 
variables of M and the local variables of its submodes. 

In the semantic definitions of the next section we model paths by stacks. 
Textually, we write stacks with the elements separated by colons and with the 
topmost element on the left. For example s = a: b : s ’ is the stack s with the top 
element a, the second element b and the rest of the stack s ’ . To show how stacks 
evolve in a pictorial way we use pattern matching. For example when we write if 
((as = a:b:as’) & (bs = c:bs’)) (as,bs) = (c:as’, a:b:bs’) we mean 
that if the current value of the stack as has topmost elements a and b and the 
current value of the stack bs has the topmost element c then the next value of 
as has discarded a and b and pushed c, and the next value of bs has discarded 
c and pushed a and b. 

3.2 Update Rounds 

In an update round, the semantic function nondeterministically chooses one of 
the modes of the resulting fiat agent and executes the discrete update on that 
mode. Using a pseudo-code like notation this can be described as shown below. 
State updateRound (Agent a, State s){ 

return forany (m in subModes(a)) discreteUpdate (m, s) ; } 

The discrete update of a mode is a sequence of enabled implicit and explicit 
transitions starting at the default entry point of the mode and ending at the 
default exit point of the mode. The algorithm for generating this sequence is 
given below. In the first step it uses the global history variable hs, that is itself 
a stack, to execute a series of default entry transitions down to the last control 
point where the explicit execution got stuck, i.e., where all the explicit transitions 
were disabled. A default entry transition restores the saved submode and point by 
popping them from the history stack and pushing them on the control stack ct. 
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State discreteUpdate (Mode m, State s) { 

Stack ct = de:m: [] ; State st = s ; //put de and m 

while (ct != dx:m:[]) { // while dx not reached 

while (ct = de : ct ’ ) //while de is the top point 

if (st.hs = pt:md:hs’) 

(ct, st.hs) = (pt : md : ct ’ , hs ’ ) ; //default entry transition 
else ct = dx;ct’; //default identity transition for leaf mode 
while (enabledFanOut (ct , st) != {}) 

(ct, st) = forany (t in enabledFanOut (ct , st)) t(ct, st) ; 
let (ct = pt:md:ct’) in 
if (pt != de) 

(ct, st.hs) = (dx:cf, (pt=dx?de Ipt) :md: st .hs) } 
return st ; } 

If the history stack hs is empty and the top point on the control stack ct is 
the default entry point de then a leaf mode has been reached and the identity 
transition of the leaf mode is executed. 

In the second step, the algorithm executes a sequence of explicit, enabled 
transitions starting at the control point obtained in the previous step and ending 
at the control point where all the explicit transitions are disabled. The enabled 
fanout of a mode reference is the set of enabled transitions in the associated 
mode definition, with source point pt and with source state st. 

In the third step, the algorithm executes an implicit exit transition provided 
that the last transition was not a self group transition (in this case, the top 
point pt is equal to de). The default exit transition saves the relative value of 
the control point from the previous step on the top of the history stack and passes 
the control to the default exit of the parent mode. Note that, if the top point on 
the control stack ct was the default exit point dx, then the exit transition saves 
on the history stack hs the default entry point de. This assures that in the next 
step, the deepest point is tried first. 

Since the top of the control stack is dx and not de, the first step is skipped 
when control is passed up to the parent mode. The second step in this case 
amounts to executing a group transition if any enabled transition exists. If this 
is not the case, the control is passed in the third step up again to the enclosing 
parent mode and so on up to the top mode. If any of the group transitions 
is enabled, then executing this transition (and possibly other), may return the 
control to the default entry point de of the mode, and the algorithm proceeds 
by skipping the third step and executing all the default entry transitions. 

3.3 Time Rounds 

In a time round, for a given state si, the semantic function executes for a time 
interval d, and produces a new state S2 = s(d), where s is any flow that is a 
solution of the active set of control constraints, not violating the current set of 
invariants and such that s(0) = si. The semantic function is shown below, where 
the type Constraints is assumed to contain a set of algebraic constraints, a set 
of differential constraints and a set of invariants. 
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State timeRound (Agent a, State s) { 

Constraints c = agentConstraints (a, s) ; 
return forany ((f, d) in solution(c, s)) f(d); } 

The set of active constraints for an agent is the union of the active constraints 
of each mode in the agent. 

Constraints agentConstraints (Agent a, State s) { 

Constraints ac = {}; 
forall (m in modes (a)) 

ac = ac U modeConstraints (m, s) ; 
return ac ; } 

For each mode, the set of active constraints is easily recovered form the history 
variable. 

Constraints modeConstraints (Mode m, State s) { 

Constraints me = getConstraints (m) ; Stack hs = s.hs 
while (hs = pt:md:hs’) { 

me = me U getConstraints (md) ; 
hs = hs ’ ; } 
return me ; } 

Hence, in a global semantics, the flows in all agents are synchronized with each 
other. 

3.4 Global Execution 

The semantic function for the execution of a hybrid agent nondeterministically 
chooses in each step either an update round or a time round, as shown by the 
following pseudo-code segment. 

State macroStep (Agent a, State s) { 

[] return updateRound(a, s) ; 

[] return timeRound (a, s) ; } 

4 Modular Simulation 

The global semantics given in the previous section can be readily implemented 
in an algorithmic way to obtain a precise simulation for any hybrid system 
described in Charon. However, such a simulation has a big disadvantage: it 
is not modular. In other words, one can not simulate the behavior of a mode 
in isolation independent of other modes or the mode hierarchy. The lack of 
modularity precludes efficient implementations. For example, all flows in the 
previous section are synchronized on the same clock. 

In this section we present an alternative, modular simulation for hybrid 
agents. This simulation may have a very efficient implementation. However, its 
disadvantage is that it only approximates the conceptually ideal solution. 

4.1 Update Rounds 

In a modular simulation, the time and the update rounds of the mode of an 
atomic agent are constructed in a modular way from the time and the update 
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rounds of its submodes. The state passed along the modes is automatically co- 
erced to the appropriate state for that mode, i.e., a mode can only access that 
part of the state that corresponds to its own variables. In programming languages 
terminology, the discreteUpdate and the timeRound functions are polymorphic. 

In the modular version we do not have to work with path prefixed variables 
and points because the structure of a hierarchical mode is not destroyed (flat- 
tened) . Moreover, in this case each mode has its own history variable, keeping a 
tuple: the last visited submode and its associated point. The modular version of 
the discrete update function is shown below. The initialization round of a mode 
is obtained by calling discreteUpdate at the initialization entry point. 
PointxState discreteUpdate (Mode m, Point p, State s){ 

Mode md = m; Point pt =p; State st = s; 



repeat { 
if (md 
(md, 
else 

(md, pt, St) 



m & pt = de) 
pt) = s.hs; 



forany (t in 



t (md, pt , St) ; 

if (md = m & pt in exitPts(m)) 
return (pt, st) ; 
else 

(pt, st) = discreteUpdate (md, pt , st) ; 
until (enabledFanOut (md, pt, st) = {}) ; } 
s.hs = (md, pt) ; //update history 

return (dx, st) ; } //done 



//loop 

/ /control is at default entry point 
//execute default entry transition 
/ /control is at regular entry 
enabledFanOut (md,pt , st)) 
//execute transition 
/ /control reached exit point 
/ / done 

/ / control reached submode 



4.2 Time Rounds 

Taking the idea of modularity seriously, in a time round each agent should be 
able to integrate independently of the other agents, and the integration inside a 
submode should be done independently of its supermodes. 

The independent integration of the subagents in a composite agent, or equiv- 
alently the integration of the top modes of the associated flattened agent, is the 
topic of the next section. In this section we are concerned with the hierarchical 
integration for a mode. The main goal is to allow the modes to integrate at 
different speeds without compromising too much the ideal solution. 

Our main assumption is that the integration speed of the parent mode is of 
an order of magnitude slower than the integration speed of the submodes. In 
this case, we may assume that the values integrated in the parent mode, remain 
constant while the submodes perform their own integration. For example, in 
Figure 2, we assume that the integration speed for x is slower than the integration 
speed for y that is also slower than the integration speed for z. This idea is shown 
algorithmically below. 

The time round function gets as input the mode, the state, the simplified 
invariants of its parent mode and the integration step of its parent mode. It 
first computes the current submodes and the set of invariants. Then it enters 
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Fig. 3. Global execution 



the integration loop. In this loop it first simplifies the invariants according to 
the variables integrated in its supermode (their values are assumed to be fixed) 
and if the loop was traversed at least once, according to the variables declared 
in this mode or above but integrated in the submodes. Then it predicts its own 
integration step. 

StatexTime timeRound(Mode m, State s, Invariants i, Time t){ 

State St; Mode md; Time d, dt; 

Invariants inv = getlnv(m) U i; //get invariants 

(md, pt) = s.hs; //get active submode and point 

for (Time tm =0; tm < t; tm = tm + dt) { //while time left 

inv = simplify(s, inv); //simplify invariants 

dt = predict(inv, s, getConstraints (m) , tm) ; //predict dt 
(st, d) = timeRound(md, s, inv, dt) ; //execute submode 

st = integrate (st , getConstraints (m) , d) ; //integrate 
if (d < dt I vioIated(inv, st, tm+d) ) 

return (st, tm + d); } //violation return 

return (st, tm) ; } //normal return 

Then it calls its current submode (known from the history variable) to execute a 
time round. It also constrains the integration time of the submode by passing its 
own simplified invariants. When the submode returns, the mode synchronizes its 
own differential variables with the differential variables owned by the submodes 
by performing the integration step. If the submode returned before the assigned 
integration time or the invariant of the mode was violated, the mode itself re- 
turns. Otherwise it returns normally. In this way, all variables are synchronized 
up to the top level. 

4.3 Global Execution 

In the modular simulation of the global execution we want to be able to in- 
tegrate each subagent of a composite agent (or equivalently each mode of the 
corresponding flattened agent) at a possibly different speed and along intervals 
of different length. This however inevitably leads to an out of synchronization 
between the agents, because as long as an agent is integrating it cannot become 
aware of the changes produced by the other agents. 
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The main idea of our approach is to keep the out of synchronization interval 
between agents bounded, even if the agents proceed with different speeds. An 
intuitive analogy would be that of a rubber band that surrounds the agents and 
cannot be expanded more than a length, say dt. 

For this purpose, each step in the global execution first picks up the modes 
with minimum and second minimum local time. For example, in Figure 3 we 
pick the modes M 2 and Mi . Then we compute the time round interval inc for the 
minimum mode such that its local time may exceed by at most dt the current 
local time of the second minimum mode. For example, in Figure 3, the increment 
is inc. 

The time round may end before the time interval inc was finished if the 
invariants of M 2 get violated. Hence, the time round returns, as shown in Figure 
3, with an actual time increment ai. In this case, the mode M 2 also executes an 
update round to synchronize the discrete variables with the analog ones. To be 
able to compute the minimum and the second minimum time values and their 
associated modes, we keep an array of current local times of modes. This idea is 
presented algorithmically below. 

Time[]xState macroStep(Time [] mTms, Agent a, State s){ 

/ /initialization 
//compute index for min. 

//compute index for second min. 
//select mode with min. time 
//compute time interval 

//execute time round 
//update the actual time for m 

//execute update round 
//make new state and time visible 

5 Conclusion 



Point p; Mode [] mds = modes (a); 
int i = getMin(mTms) ; 
int j = get 2ndMin (mTms) ; 
m = mds [i] ; 

Time inc = mTms [j] -mTms [i] +dt ; 
(State s, Time ai) = 

timeRound(m, s, {}, inc); 
mTms [i] = mTms [i] + ai ; 
if (ai < inc) (p, s) = 

discreteUpdate (m, de, s) ; 
return (mTms, s) ; } 



In this paper, we have presented a language for specification of hybrid systems 
that supports concurrency and hierarchy in a modular fashion. We hope that 
Charon is rich enough to support high-level modeling of embedded software, 
and is formal enough to support analysis. In this paper, we have proposed only a 
high-level outline for developing a modular simulator. We need to explore three 
orthogonal issues. First, finding a solution to a set of differential and algebraic 
constraints in presence of invariants requires careful detection of boundary cross- 
ings (see, for instance, [14]). Second, we handle concurrency by allowing agents to 
integrate separately based on their local clocks. When the guards and invariants 
of one agent depends on the values updated by the other agents, such a scheme 
may require detection and rollback. This is closely related to well understood 
problems concerning global states in distributed systems (see, for instance, [7]). 
Third, choosing different time scales for solving constraints at different levels of 
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the mode hierarchy requires good heuristics to predict the step sizes. This can be 
done, in principle by determining the singular values of the linearized equations 
and scaling the equations appropriately. However, choosing a simple implicit in- 
tegration scheme guarantees numerical stability and acceptable results, albeit 
with poor efficiencies [14]. 
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Abstract. In this paper we describe an experimental system called 
d/dt for approximating reachable states for hybrid systems whose con- 
tinuous dynamics is defined by linear differential equations. We use an 
approximation algorithm whose accumulation of errors during the con- 
tinuous evolution is much smaller than in previously-used methods. The 
I d/dt] system can, so far, treat non-trivial continuous systems, hybrid 
systems, convex differential inclusions and controller synthesis problems. 

1 Introduction 

The problem of calculating reachable states for continuous and hybrid sys- 
tems has emerged as one of the major problems in hybrid systems research 
[G96,GM98,DM98,KV97,V98,GM99,CK99,PSK99,HHMW99]. It constitutes a 
prerequisite for exporting algorithmic verification methodology outside discrete 
systems or hybrid systems with piecewise-trivial dynamics. For computer scien- 
tists it poses new challenges in treating continuous functions and their approx- 
imations and in applying computational geometry techniques to problems in 
higher dimensional spaces. For control theorists and engineers the problem sug- 
gests a fresh way of looking at systems with under-specified inputs and increases 
their awareness to some practical computational aspects of controller design. 

In this paper we describe an experimental system called d/dt which can 
approximate reachable states for hybrid systems whose continuous dynamics is 
defined by linear differential equations. The performance is much better than 
the more general method of “face-lifting” we have used in the past [DM98] . 

The rest of the paper is organized as follows. In section 2 we define the prob- 
lem of calculating reachable states and suggest a general procedure which solves 
it iteratively. The basic computation step of the procedure cannot be performed 
exactly and in section 3 we describe an over-approximation scheme for linear sys- 
tems, having the advantage of not propagating errors from one step to another. 
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Extensions of the algorithm to deal with hybrid systems, controller synthesis and 
continuous disturbances are described in section 4 along with several examples. 



2 The Basic Problem 

Let T = M_|_ be a time domain, let X he & bounded subset of M" and consider a 
continuous dynamical system A over X defined by the equation x = /(x). We 
use the notation x — ^ x' to indicate that the solution a of the equation with 
X as an initial condition satisfies a\t] = x'. In words we say that x' is reachable 
from X in time t. 

Definition 1 (Successors). Let A be a dynamical system defined by x = /(x). 
The successor operator 6 : 2^ ^ 2^ is defined for a subset F of X and an 
interval I CT as: 

Si{F) = |x' : 3x G F G / X x'} 

We use the notation Sr for 5[r,r] (states reachable after exactly r time), 5 for 
'^[o,oo) (all states reachable after any non-negative amount of time) and i5/(x) for 
i5/({x}). Note that 5 has the semi-group property, i.e. 5i^{5i-^{F)) = Sj^ 
where 0 is the Minkowski sum, and in particular <5[o,r2] ('^[o,ri](^)) = '^[o,ri-i-r 2 ] (F). 
In certain cases when the differential equation admits a closed-form solution, 
one may characterize S{F) symbolically by a formula and then try to obtain a 
closed-form solution by quantifier elimination. However, this works in rather ex- 
ceptional cases (see for example [CV95,PLY99]). Instead we propose a numerical 
algorithm which works by discretizing time into multiples of a fixed time step r. 
The abstract algorithm for calculating 6{F) is the following: 

Algorithm 1 (Exact Calculation of S{F)) 



pO p 

repeat 

pk+i _ pfeU(5[o,r](P'=) 

until 



In order for a function to be computable by a discrete device its domain and range 
need an effective representation as well as an effective and terminating procedure 
which takes the representation of any element of the domain and transforms it 
to a representation of its image by the function. For example, functions over 
the integers can be computed by applying well-known algorithms for addition 
and multiplication to unary, binary or decimal representations of numbers. The 
mathematical real numbers pose a special problem in this respect, a problem 
which we do not address here but assume to be solved for all practical purposes. 
Our main concern here is to compute functions over subsets of X. From this 
perspective Algorithm 1, when applied exactly suffers from the following two 
problems: 
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1. The exact calculation of <5[o,r] is not more feasible than the calculation of the 
whole 6. 

2. Even if i5[o,r] was computable, the algorithm usually does not terminate after 
a finite number of steps. 

To overcome these problems we resort to approximate calculation of i5[o,r] &nd 
S. In order to be effective, i.e. to do any computation at all, we can replace 2^ 
by a countable and effectively enumerable subset C whose union gives X, e.g. 
the set of all polyhedra with rational vertices. Elements of 2^ not in C are thus 
either under- or over-approximated (see Figure l-(a)). The type of approximation 
which is used depends on the problem to be solved. If we want to characterize all 
the possible behaviors starting from a given initial set, an over-approximation is 
used. If we want to characterize the set of states from which a property can be 
satisfied, under-approximation is preferred. 

An effective approximation of Algorithm 1 can thus be implemented by re- 
placing all the operations (Boolean operations, equivalence testing and calcu- 
lation of <5[o,r]) by their approximated versions.^ If the approximate algorithm 
terminates, the result is an over-approximation of S{F). 






(a) (b) (c) 

Fig. 1. (a) A set F and over- and under-approximated by polyhedra. (b) The 
same set approximated by orthogonal polyhedra. (c) Accumulation of errors in 
nave approximate computation. 



The termination of the procedure, however, cannot be guaranteed since there 
are infinitely many polyhedral sets. Moreover, the implementation is very com- 
plicated because the sets can be very complex non-convex polyhedra for 
which there is no useful canonical form and the test is very ex- 

pensive. Hence we restrict further the class of sets to be what we call griddy 
polyhedra, i.e. 2® where B is the set of all closed unit hypercubes with integer 
leftmost corners. Using this finite class of sets guarantees convergence of Al- 
gorithm 2 (provided we restrict our analysis to bounded domains) and allows 

^ Note that if the class C is closed under Boolean operations, only 5[o,r] (F) needs to be 
approximated. This holds for arbitrary polyhedral sets but not for convex polyhedra 
or ellipsoids. 
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us to benefit from a relatively-efficient canonical representation for both convex 
and non-convex sets [BMP99], supported by an experimental software package. 
The price, however, for using orthogonal polyhedra is that the quality of the 
approximation they provide in terms of Haussdorf distance per vertex is poorer 
than that of arbitrary polyhedra (zero-order vs. first-order in the approximation 
jargon) but such a compromise seems unavoidable. 

A nave approximate version of Algorithm 1 is guaranteed to converge to a 
superset of S{F) after finitely many steps. However, the distance between the 
result and 6{F) might be too big for the result to be useful. The reason is that 
over-approximation errors accumulate dramatically as illustrated in Figure l-(c) 
where we try to calculate successors of the set D. Since x' is reachable from x 
we must include the whole box D' in the set of successors. This box contains 
points such as y not really reachable from D, which bring in the next iteration 
new points, such as y', and we end up adding boxes such as Z?" which are not 
reachable from D at all. This over-approximation error can propagate fast and 
the result might cover the whole space unless some hardly-formalizable hacking 
is used [DM98,GM99]. Similar phenomena are exhibited, for example, in abstract 
interpretation of programs over the integers [CC92] where over-approximation 
is called widening. This is why there is not much hope in finding finite quotients 
of continuous systems, except for special cases such as timed automata [AD94]. 

Here we need to find the right compromise between the desire to converge and 
the accumulation of errors. We propose a method, specialized for linear systems 
of the form x = Ax which achieves this trade-off. The basic idea here is to 
separate the accumulation and storage of states reachable in one step (and those 
must contain an approximation error) from the computation of states reachable 
in the next step (see also [GM99]). The main attraction of this method compared 
to traditional ways to treat linear systems is in its adaptability to hybrid systems 
and to systems with under-specified input. 



3 The Approximate Method for Linear Systems 

Let conw({xi, . . . , Xm}) be the convex hull of a set of points, i.e. {x : x = 
AiXi -I- • • • , AmXm} for non-negative Xi whose sum is 1. For linear systems we 
have <St(x) = e"^*x and the matrix exponential, as a linear operator, preserves 
convexity: 



St{conv{{xi , . . . ,x„})) = conv{{St{xi ), . . . ,<5t(x„)}). 

This means that for a convex set F = conv(V) where V = {xi,...,Xm}, 
and for every t, the states reachable from F can be determined by the states 
reachable from V (see Figure 2- (a)). We exploit this property to approximate 
i5[o,r] (conv(V)) based on the set of points V U (5r(V) where <5r(V) is computed 
from V by a finite number of matrix exponentiations or numerical integration 
steps. Our approximation scheme consists of three steps: 
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1. Compute G = conw(V, <5r(V)) (see Figure 2-(b)). This set is an approx- 
imation of <5[o_r](conu(V)) but neither an over-approximation nor under- 
approximation. The convex-hull algorithm provides us with information con- 
cerning the orientation of the faces which is used in the next step.^ 

2. Push the faces of G outward to obtain a bloated convex polyhedron G' 
which is guaranteed to contain the required set (Figure 2-(c)). The amount 
of pushing is determined by the time step r and the matrix A (see the analysis 
in the appendix). Pushing inward will result in an under-approximation. 

3. Over-approximate G' by a griddy polyhedron ^](F’) (Figure 2-(d)). 

The approximate algorithm for calculating S{F) for F = conv(V) is defined 
below: 

Algorithm 2 (Approximate Calculation of S(F) for Linear Systems) 



po ■- F; v° := V; k:=0; 
repeat 
k := k + 1; 

:= <5r(V''“^); 

G^ := conu(V'=-i U V'=); 
G'^ := bloat{G’^); 

G^ := griddy{G^); 

pk pk—1 ^ f^k 

until 



There are two types of errors accumulated in the process of calculating P^: 
from the actual set to its bloated convex hull and from there to the griddy poly- 
hedron. However these errors do not propagate to the next step which computes 
based on U and not on P^ (Figure 2-(e)). Recall that our or- 
thogonal polyhedra package [BMP99] maintains <5|q 2r](^) ^ single canonical 

object and not as a union of convex polyhedra or ellipsoids (Figure 2-(f)). The 
algorithm can be fine-tuned by changing the time step r and the size of the 
hypercubes. 

Result 1 (Computation of Reachable States for Linear Systems) There 
exists an implemented algorithm for over -approximating the reachable sets of sys- 
tems defined by linear differential equations. 

The reason this result is not a theorem is due to the following facts: 

1. There is always a trivial over-approximation of any subset F of X, namely 
X itself. 

2. The smallest polyhedral or griddy set which contains 5{F) is as impossible 
to compute as 5{F). 



^ We use the convex- hull algorithm supplied with the LEDA library [MV99]. 
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(e) (f) 



Fig. 2. (a) A set F = conv{{xi,X 2 }) and its exact successors for time inter- 
vals [0,r] and [r, 2r]. (b) Approximating 6[Q^r]{F) by convex hull, (c) Bloat- 
ing the convex polyhedron to obtain a polyhedral over-approximation, (d) 
Rectangulating the polyhedron into (e) Repeating the same proce- 
dure in the next time step to obtain (^) "bhe accumulated states 
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3. Like in many other numerical problems, the best upper-bounds which can be 
easily proved on the approximation error are much larger than what happens 
in practice. 

So let us be content with the fact that the method gives reasonable approx- 
imation in rather short time. So far we were able to calculate rather easily the 
reachable states of non-trivial systems with up to 6 dimensions (in fact, the mea- 
sure of complexity for such problems depends on the dimensionality, the coupling 
of the variables and the granularity of the discretization). Figure 3 shows the 
states reachable from 



F= [0.025,0.05] X [0.1,0.15] x [0.05,0.1] 



by the 3-dimensional system defined by 





(-1.0 -4.0 


0.0 


A= 1 


4.0 -1.0 


0.0 




O 

O 

O 

o 


0.5 




Fig. 3. Calculating reachable states for a 3-dimensional system. 



4 Extensions and Applications 

4.1 Piecewise-Linear Systems 

For purely continuous linear systems there are classical methods, more efficient 
than ours, for solving certain problems such as stability or controller synthesis. 




Approximate Reachability Analysis of Piecewise-Linear Dynamical Systems 



27 



However the main advantage of our approach is manifested in the analysis and 
controller synthesis for linear hybrid automata which may switch between several 
“modes” and hence define piecewise-linear dynamical systems. We demonstrate 
the adaptation of our method to such systems informally using the hybrid au- 
tomaton of Figure 4, which consists of two continuous variables, and two discrete 
states. In each discrete state the continuous variables evolve according to the 
corresponding linear dynamics and when some switching conditions (transition 
guards) are satisfied, the system moves from one state to another. 

Starting from an initial set (qo,F) the reachable states are calculated as 
follows: we apply our procedure to F with the Aq dynamics and calculate forward 
(5°(F). Then we calculate the intersection of the result with the guard to obtain 
a set F', move to state qi with F' as the set of initial states, calculate S^(F') 
and so on and so forth. This method is similar to the one used in tools such 
as KRONOS [DOTY96] for timed automata and HyTech [HHW97] for hybrid 
automata with constant derivatives [ACH+95]. 

The main technical difficulty in applying our vertex-based approximation 
technique to such systems is that not all trajectories departing from the vertices 
reach a transition guard simultaneously (some may not reach it at all) . Hence we 
have to calculate S{F) and intersect it with the guard to obtain the new initial 
set. Unfortunately, this set is already an over-approximation and, moreover, 
it might have many vertices and the reduction of their number might require 
further approximation. The bottom line is that we can avoid propagation of 
over-approximation errors during the continuous evolution but not while doing 
transitions. 

An example run of d/dt on the hybrid automaton of Figure 4 where 



/-2.0 -3.0\ 
3.0 -2.0 J 



and A I 



fO.O - 0 . 6 \ 
^3.0 0.0 J 



and the initial set is F = {go} x [0.3, 0.6] x [—0.2, 0.2], appears in Figure 5. 
Initially the successors by Aq (a “center” dynamics) are calculated until they 
all intersect the guard x\ < —0.15 (a). Then dynamics A\ is applied, shrinking 
the set until intersection with the guard xi > —0.02 (b). From this guard the 
dynamics Aq induces a “ring” of states which stay in go forever (c). 




Fig. 4. A hybrid automaton. 
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<70 <7i <70 

(a) (b) (c) 

Fig. 5. The 3 stages in the calculation of S{F) of the hybrid automaton of 
Figure 4. 



4.2 Under- Approximation, Backward Reachability and Control 



The 6 operator is a basic ingredient in forward reachability analysis. Other ver- 
ification and synthesis problems require different variants of this operator. 

The reader might have guessed that calculating under-approximations is done 
by a slight variation of the algorithm, i.e. pushing the faces of the polyhedron 
inside and finding an orthogonal under-approximation. Backward reachability, 
that is, finding all the points from which a set F is reachable can be performed 
by computing 6 for the reversed system x = —Ax. 

For the purpose of controller synthesis for hybrid systems [ABDPMOO] we 
need an under-approximation of the “F Until G” operator, which returns the 
points from which you can stay within the set F either forever or until you 
reach a set G (which is typically the guard of a transition to another state). A 
similar operator is needed for analyzing hybrid systems with invariants. Consider 
F = [-0.1, 0.1] X [-0.030.1], G = [0.02, 0.06] x [-0.05, -0.02] and a dynamics 



A = 



f-0.5 4.0\ 
V-3.0 -0.5 J 



The two parts of F Until G, as calculated by d/dt 



appear in Figure 6. 



4.3 Continuous Disturbances 

Consider systems of the form x = Ax-|-i?u where u ranges inside a convex set U. 
It has been suggested in [V98] to use the maximum principle from optimal control 
to find Sr{F) of a convex set F = conv(V) under all possible input signals. We 
have implemented this procedure and incorporated it into our system. We have 
tested it on a 4-dimensional example adapted from example 4.5.1 of [KV97], pp. 
279-285, where ellipsoids are used instead of polyhedra. The system is defined 
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Fig. 6. The Until operator: (a) The states which can stay in F forever, (b) The 
states which can stay in F until reaching G. 



by: 



/ 


0.0 


1.0 


0.0 


0 . 0 \ 




-8.0 


0.0 


0.0 


0.0 




0.0 


0.0 


0.0 


1.0 


V 


0.0 


0.0 


-4.0 


0.0 / 



F= [0.02.0] X [-1.01.0] X [0.0, 2.0] x [-1.0, 1.0] 



u= [-0.5, 0.5] X [-0.005,0.005] x [-0.5, 0.5] x [-0.005,0.005] 

In Figure 7 one can see the evolution of the projection on dimensions 3 and 4 over 
time, similar to the results in [KV97]. Further work on these technique might 
suggest effective methods for approximate strategies for differential games. 




Fig. 7. The evolution of a 4-dimensional convex differential inclusion over time 
(projected on dimensions 3 and 4). 
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5 Discussion 



In this work we have advanced the state-of-the-art in computer-aided reachabil- 
ity analysis for continuous and hybrid systems. We have implemented the tool 
d/dt and tested it over reproducible non-trivial examples. We are currently 



investigating various improvements and studying the trade-offs between accu- 
racy and computational efficiency. We hope that such techniques and tools will 
be used in the future by control engineers. 
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Appendix: Conservative Approximation 

As we have already mentioned when describing the approximate method for 
linear systems, the set G = conv(V,Sr(V)) is not an over-approximation of 
i5[o,r] (conv(V)) and should be replaced by its e-neighborhood (or something big- 
ger) in order to become such an over-approximation. Here we calculate the e 
that should be used. 

Consider an arbitrary point po G convfV) and a trajectory pt starting from 
this point. We have Pr = e^^po. This point belongs to i5r(V) and hence to G. By 
convexity so does all the line segment [poiPr]- Let us estimate now the distance 
between points of the true trajectory pt for t G [0,r] and this line segment. In 
fact Pt may be approximated by linear interpolation between po and Pr- The 
result of this interpolation is 

t 

Pt = Po+ -[Pr - Po), 0<t<r 
r 

and by construction it belongs to the segment [po,Pr]- The error of this interpo- 
lation can be written as follows: 

e[Po,t) = \\pt-Pt\\ = \\po + - I)Po - e*^po\\. 

Since 

1 “ 1 

et^ = I + At+ -f y 

2 ^ n! 

i—3 

and 0 < t < r we find after obvious simplifications the bound of the error: 

oo 

e[po,t) < e = M-||A||V + ^P||V, 

i=3 

where M is a constant bounding the norm ||po||- 

Hence, for every r, one can find a e = O(r^) such that all the points reachable 
from conv(V) in time r are in e-neighborhood of G. In order to over-approximate 
the set we just replace G by its e-neighborhood. 
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Abstract. The specification for the idle control problem for automotive 
engines is to maintain the crankshaft speed within a given range in the 
presence of load changes. A new cycle-detailed hybrid model of the engine 
that captnres well the interactions between the discrete phenomena of 
torqne generation and spark ignition, and the continuous evolution of 
the power-train and air dynamics, is proposed. The idle control problem 
is formalized as a safety specihcation problem on the hybrid system. The 
Tomlin-Lygeros-Sastry procednre [12] is applied to compute the maximal 
controlled invariant set that satisfies the safety specification. 

1 Introduction 

The synthesis of a control strategy for an internal combustion engine in the idle 
regime is one of the most challenging problems in engine control. The objective 
is to maintain the engine speed as close as possible to the value that minimizes 
fuel consumption, while preventing the engine from turning off when a sudden 
load variation occurs. Load variations come from two sources: (1) from devices 
powered by the engine, such as the air conditioning system and the steering 
wheel servo-mechanism, or (2) from the driver changing the inertial load when 
operating the clutch pedal. A survey on different engine models and control de- 
sign methodologies for the idle control is given in [8]. Both time-domain (e.g. [5]) 
and crank-angle domain (e.g. [13]) average-value models have been proposed in 
the literature. Several control design techniques have been applied to the idle 
control problem, such as multivariable control [10], control [5], iLoo control [6], 
/x-synthesis [7], sliding mode control [9] and LQ-based optimization [1]. 

In this paper, the idle control problem is specified as the one of keeping 
the crankshaft speed within a specified range, robustly with respect to load 
changes. The adoption of a hybrid formalism allows us to describe the cyclic 
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behavior of the engine, thus capturing the effect of each spark ignition on the 
generated torque, the interaction between the discrete torque generation and the 
continuous power-train and air dynamics, and the discrete changes in the power- 
train. The torque that is generated by each cylinder and applied to the engine 
crankshaft can be assumed to be a function of the spark ignition time, and of the 
air-fuel mixture mass loaded in the cylinder during the intake phase. Since the 
air-to-fuel ratio is assumed to be constant (at the stoichiometric value), then the 
mixture mass is controlled by the throttle plate position and is subject to the 
dynamics of the cylinder filling due to the intake manifold. Hence, the available 
controls for the idle problem are: the spark ignition time and the position of 
the throttle valve, which regulates the air inflow^ . The problem of maintaining 
the crankshaft speed within a given range is formalized as a safety specification 
for the hybrid closed-loop system. A safety specification is a state-invariance 
property, specifying a set of good states within which the closed-loop system 
must remain. A systematic procedure for computing the maximal safe set has 
been recently proposed by Tomlin, Lygeros, and Sastry [12]. This set consists of 
all the hybrid states for which there exists a hybrid control strategy (the maximal 
controller) that maintains the state in the set of good states forever, in spite of 
any discrete and continuous disturbance. The procedure is not guaranteed to 
terminate in a finite number of steps. 

By applying this procedure to the hybrid model of the engine, the maximal 
safe set for the idle control is determined. We also obtain as a by-product the 
entire set of possible controllers that satisfy the constraints. We are free to choose 
among them the ones that optimize some criteria of choice. Moreover, considering 
the amount of load torque as a parameter, we can determine the maximum value 
for which a non empty maximal safe set (and, hence, at least one controller that 
satisfies the constraints) exists^. For parameters corresponding to commercial 
cars, the procedure has terminated in a few steps (typically six). 

To summarize our main contribution, the use of a hybrid framework, where 
discrete and continuous signals are modeled in a separate but integrated man- 
ner, is a definite advantage over other approaches since it allows us to solve 
exactly the control problem while other approaches, where the system is ap- 
proximated by either continuous [11], or discrete sampled [13] representations, 
obtain approximate solutions. The paper is organized as follows: in Section 2, a 
description of the engine in the idle region of operation is offered and its hybrid 
features are exposed. In Section 3, a hybrid automaton model of the engine for 

^ The effect of a spark command on the torque generation is more visible than the 
one of a throttle plate command, since air inflow is subject to both the manifold 
dynamics and the delay due to the mix compression. Hence, sudden loads can be 
much better compensated with spark ignition than with air inflow, while air inflow 
can be used to control the engine in steady state. For simplicity, we do not consider 
the throttle valve actuation dynamics. 

^ Butts et al. [5] solve a sort of dual problem: given a bounded torque load accessible 
to measurement, synthesize a robust £i controller for a discrete-time model of the 
engine that minimizes the excursion of the crankshaft speed for the system initially 
at rest. 
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Fig. 1. The engine blocks and their communication topology. 



the idle regime of operation is proposed for the first time. In Section 4, a general 
procedure for the calculation of the maximal controller is reviewed. In Section 5, 
the procedure is specialized to the idle control problem and some experimental 
results are described. 

2 Description of the System 

The overall system is composed of three main interacting blocks, namely the 
intake manifold, the cylinders and the power-train, as depicted in Figure 1. The 
manifold pressure p depends on the throttle valve angle a and determines the 
mass of air-fuel mixture m loaded by the cylinders. The torque T generated by 
the cylinders depends on both the mass m and the spark ignition time. Finally, 
the power-train dynamics and the crankshaft revolution speed n, controlled by 
the generated torque T, are subject to the sum of load torques T; and the clutch 
position. In the sequel a detailed description of each block is reported. 

Intake Manifold Dynamics. The mass of mix m entering a cylinder during the 
intake run is assumed to be proportional to the intake manifold pressure p at 
the end of the intake run. The pressure p is controlled by a throttle valve which 
changes the effective section of the intake manifold: 

p{t) = app{t) + bpa{t) . (1) 

To prevent the choice of undesirable control laws that produce large excursions 
of the throttle valve, the throttle angle a{f) is constrained to belong to [0, 
with = 20°. 

Cylinders. In a 4-stroke combustion engine each cylinder cycles through the 
following four runs: intake (/), combustion (C), expansion (E) and exhaust (H). 
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Fig. 2. Phases of a single cylinder: dashed lines denote transitions occurring 
when 9c = 180, solid lines denote transitions occuring when a spark ignition is 
given. 



Ideally, spark ignition should occur exactly when the piston reaches the top 
dead center (TDC) configuration of the compression stroke. However, since com- 
bustion takes non-zero time to complete, it is convenient to produce a spark 
before the piston completes the compression stroke {positive spark advance), to 
achieve maximum fuel efficiency, i.e. the maximum torque generation. When a 
small value of torque is required, the spark can be ignited after the piston has 
completed the compression phase and is in the expansion stroke {negative spark 
advance). 

Let 6c denote the piston position, between two successive dead points, ex- 
pressed in terms of the angle described by the crankshaft, obtained by the inte- 
gration of the crankshaft velocity and by resetting 9c to 0° when 9c = 180°. The 
spark advance 9g, defined as the angular distance from the TDC of the com- 
pression stroke, determines the crankshaft angular position at which the spark 
is given. It is positive for sparks given before the TDC {9,s = 180° — 9c{t.s), 
where tg is the ignition time), and negative otherwise {9g = —9c{ts))- At the idle 
crankshaft speed, due to technological constraints, the feasible spark advances 
are —15° < 9g < 20°. If the spark is ignited during the compression stroke, then 
C is split into two phases, namely BS (before spark) and PS (positive spark). 
Instead, if the spark is ignited during the expansion stroke, then E is split into 
two phases, namely NS (negative spark) and TG (torque generation). In the 
first case, the expansion stroke is represented by phase TG, while, in the second 
one, the compression stroke is represented by BS. The behavior of each piston 
is then characterized^ by the six phases I, BS, PS, NS, TG and E, as shown 
in Figure 2. 

When considering a 4-stroke internal combustion engine with four cylinders, 
the kinematics of the engine is such that, at any time, each cylinder is in a 

® When the spark ignition is synchronous with the TDC, we assume the cylinder leads 
from BS to TG through the phase PS. 
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different stroke of the cycle. Since we assume that all the cylinders behave in 
the same way, then we can cluster all quadruples of cylinders’ phases in only 
three engine phases, because it does not matter which cylinder is in a certain 
phase. Then, according to the ignition constraints Qc G [160°, 180°], in PS, and 

G [0°, 15°], in NS, there are only three valid cylinder configurations and the 
discrete behavior of the system can be described by introducing the following 
three modes S = {I, BS, TG, H), S+ = (/, PS, TG, H), S- = (/, BS, NS, H). 

The transitions between S, 5'+ and S- are characterized as follows. In phase 
S, the cylinder in expansion is generating a torque (TG), and the cylinder in 
compression has not yet received the spark command (BS). If a spark ignition 
occurs before the end of the compression run, then the cylinder that is still in 
compression enters phase PS, which corresponds to the transition from S to 
S+. Otherwise, at the TDC, the expansion phase starts (NS) and the transition 
from S to S- takes place. In phase 5'+, the spark command has been given for 
the cylinder in compression (PS), while the cylinder in expansion is generating 
a torque (TG). At the TDC, the cylinder which was in compression starts the 
expansion run entering phase TG, which corresponds to the transition from S.\. 
to S. In phase S-, the cylinder in expansion is waiting for the spark command 
(NS), and the cylinder in compression has not received the spark command yet 
(BS). It is worth noting that no torque is generated in this case. When the spark 
ignition is given, the cylinder which is still in the expansion run changes from 
NS to TG, and the transition from S- to S takes place. 

The evolution of the torque, generated by each piston during the expansion 
phase, depends on the thermodynamics of the air-fuel mixture combustion. To 
simplify the model, we represent by T the average value of the torque generated 
over the expansion phase. Such value is proportional to the air m loaded in 
the cylinder during the intake phase and to the ignition efficiency (increasing) 
function rj(6s) < 1. 

Since there is a delay from the time the air mass m is trapped in the cylinder 
and the time the torque is generated, the amount of loaded air mass must be 
stored for each cylinder. To this end, we introduce two variables, me and me 
denoting, respectively, the mass of air trapped at the end of the intake run in 
the cylinder starting the compression run, and the mass of air trapped at the 
end of the compression run in the cylinder starting the expansion run. Hence, 
the torque T produced by each piston is expressed as 

rp _ i Gv((^s) Trie for positive spark advance, . 2 'v 

\Grj(6s)mE for negative spark advance. ' 

The torque T(t) generated by the engine is obtained by applying a zero order 
hold block to each cylinder output T, and summing all the piecewise constant 
contributions of the pistons. 



Power-train Dynamics. When the clutch is pushed, under the action of the 
torque T(t) generated by the engine, the crankshaft speed evolution is deter- 
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mined by the following mechanical equations 

h{t) = ann{t) +bn{T{t) -Ti{t)) (3) 

dc{t) = kcn{t) (4) 

where a„ = —B/J and = 1/ J, with J and B denoting the inertial momentum 
and the viscous friction coefficient of the segment of the power-train from the 
crankshaft to the clutch, respectively. If 9c is in degrees and n is in revolutions 
per minute, then kc = 6. When the clutch is released, the coefficients a„ and 
are replaced by = —B/{J+J') and = 1/(J-|- J'), where J' denotes the 
inertial momentum of the primary drive-line. 

Finally, the torque load T/ is assumed to belong to [0, where the value 

ymax jg |^]- 0 ated as a parameter for the control problem. 

3 Hybrid Model of the Engine 

We model the mixed discrete-continuous dynamics of the engine as a hybrid au- 
tomaton. We consider a particular class of hybrid automata characterized by a 
set of discrete locations (also called modes) corresponding to the FSM states, a set 
of continuous variables and a set of piecewise-constant variables. The controller 
and the environment act on the system with two kinds of inputs: the continuous 
inputs affect the continuous dynamics; the discrete inputs determine the dis- 
crete mode transitions, the resetting of continuous variables and the setting of 
symbolic constants. This modeling formalism combines the features of [2] with 
elements of the hybrid dynamics of [12]. The formal definition and the behavior 
of this hybrid automaton is analogous to the one described in [4], with the sep- 
aration between continuous variables and piecewise-constant variables explicitly 
introduced here. The hybrid model of the 4-stroke 4-cylinder internal combus- 
tion engine has six modes S-, S, S+, S^, , and S^, derived from the three 

modes S - , S, and S+ of the four cylinders and the two discrete positions of the 
clutch, which can be either closed or open. Figure 3 shows the resulting hybrid 
automaton. Hence, we can formally write the engine hybrid automaton as a tu- 
ple H = {{Q,X,S}, {Sc,D}, 

where: 

• the state space is composed of the finite set of modes or locations Q, 
which consists of S-,S,S+,S^,S^,S^, the space of continuous variables X = 
{{p,n,9c) I (p,n,6c) € M^}, and the space of piecewise-constant variables S' = 
{{T,mc,mE,9s) \ {T,mc,rnE,9s) G M^}. An element (q,x,^) in the space 
Q X A X S is called a configuration; 

• the control inputs a{t) and (Tc can be described by means of the domain 

of continuous input values U = [0,a'"“'^], the finite domain of discrete control 
events Xc = {spark} with E}. = XcU {e} being the set of discrete eontrol moves 
and the special e move being the silent move, the discrete controller feasible 
move function : Q x X x Ei ^ 2^« \ {} described as follows"^: 
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p{t) = Opp{t) + bpa{t) 




p(t) 


= a„p(t) + b„a(t) 




p{t) = Opp{t) + bpa{t) 


n[t) = — b^Ti[t) 




n{t) 


= al:n(t) + bl:(T(t) - T,(t)) 




h[t) = a^n[t) + b^[T[t) — Ti[t)) 


9c{t) = kc7l[t) 




edt) 


= kcTl[t) 




9c{i) = kc7i[t) 



(o-f, CTe) = (e, e) (°'ci “"e) - (<^i <^) (CTf, CTe) = (e, e) 




p9) 


= a„p(t) + 




p(t) = a„p(t) + b„a(t) 




p(t) = a„p(t) + b„a(t) 


n{t) 


= anTl[t) — bnTi[t) 




fi[t) = a„n(t) + bn{T[t) — Ti[t)) 




fi[t) = anTi[t) + bn{T [t) — Ti[t)) 




= kcTl[t) 




9^{t) = k^n[t) 




9^{t) = kcn[t) 



Fig. 3. Hybrid model for the engine running at minimum. 



Mf^%S+,9c < 180°) = {e} 
Mf^%S-,0c < 15°) = {e, spark} 

= 15°) = [spark] 

< 160°) = {e} 

Mf"'=(S',160°< 9c< 180°) = {e, spark] 



Mfsc 

Mfsc 

j^disc 

Mdisc 



{S^,9,< 180°) = {e} 

{S^,9c < 15°) = {e, spark] 

\s^,9c = 15°) = [spark] 
{S^,9,<l00°) = [e] 

(S^, 160° <9c< 180°) = [e, spark} 



and the continuous controller feasible move function : QxA'xS'— >2°^\{} 
described as follows: Mf^{q,x,^) = {a | a G [0, amax = 20°]}, V {q,x,£,)] 

• the disturbance inputs T[(t) and Ue can be described by means of the 
domain of continuous disturbance values D = [0,T“’^’^j, the finite set of discrete 
disturbance events Eg =[on, off, run, run-on, run-off] (where the events on 
and ojf represent opening and closing the discrete position of the clutch, the 
event run represents reaching the boundary 9c = 180° for the continuous state, 
the events run-on and rumojf represent the simultaneous occurrence of a clutch 
operation and reaching the boundary, and S], = EcU [e] is the set of discrete 

^ Notice that = 15°) = [spark] is a discrete control move required when 

the spark was not given yet and must be given now, since it is the last valid ignition 
time instant. 
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disturbance moves), the discrete disturbance move function : QxXx ^ 

\ {} described as follows®: 



Mf-%S+,0,<l8O°) = {e,off} 

= 180°) = {run, run.ojf} 
Mf^%S-,e,<15°) = {e,off} 

< 180°) = {e, off} 

9c = 180°) = {run, run.off} 



Mf^%S{^,9c < 180°) = {e, on} 

Mf "‘=(5:®, 9c = 180°) = {run, run.on} 
Mf^‘^{Sf,9c < 15°) = {e, on} 
Mf^<^{S^,9c < 180°) = {e, on} 

9c = 180°) = {run, rumon} 



and the continuous disturbance feasible move function: : Q x X x ^ 

2^ \ {} described as follows: Mf^{q,x,f) = {T/ | T, G [0, V {q,x,0; 

• the transitions are described hy f:QxXxSxUxD^ M" which 
models the time-invariant continuous dynamics, which depend on the mode® and 
the transition function S : Q x X x Ei x S}. x ^ \ {} modeling the 

discrete dynamics, as depicted in Fig. 3. 



4 Synthesis of Hybrid Static State Feedback Controllers 

The engine control problem at hand belongs to the class of safety problems. A 
safety property V asserts that nothing “bad” happens along trajectories and can 
be expressed by specifying a subset Good of the configuration space {Q x X x 
E). The co-set of Good is called Bad. The hybrid automaton H, with initial 
configurations {Q x X x S')o C Good, is safe with respect to the safety property 
V if there exists a control strategy that guarantees all its trajectories that start 
in (Q X A X S")o remain within Good. The maximal safe set. Safe, is the maximal 
subset (QxAxS')oof Good for which the hybrid automaton H is safe with 
respect to V , i.e., the maximal robust-controlled invariant set of configurations 
contained in Good. The maximal controller is the class of all the hybrid static 
state-feedback control strategies that guarantee that all the trajectories starting 
in Safe remain within Good. 

For the hybrid automaton described in Section 3, we define Good as the set 
of configurations for which the crankshaft speed is within the range [770,830], 
i.e.. Good = {(g, a;,^) & Q x X x E \ 770 < n < 830}. The design of a controller 
requires the computation of the maximal safe set Safe. 

Gomputing the maximal safe set [12]. This set is obtained by first overapproxi- 
mating it with all the good configurations. Then all configurations are obtained 
from which the environment can drive the system into an unsafe configuration 
via either one discrete jump, or one continuous flow. These are the configura- 
tions from which the environment can push the system into Bad in one “step”, 

® Notice that Mf‘“={S+,9c = 180°) = {run, run.ojf }, Mf“^{S+,9c = 180°) = 
{run,run.on}, Mf“^{S,9c = 180°) = {run, run.ojf } and Mf‘“^{S^,9c = 180°) = 
{run, run.ojf} model discrete moves forced by the continuous state. 

° We specify the continuous dynamics / by defining functions fq-.XxSxUxD^X 
for each q £ Q. The functions fq, as specified in Figure 3, are taken from (1), (3) 
and (4). 
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and should be avoided by the controller. One iterates this computation, finding 
successively the configurations from which the environment can push the system 
into Bad in i steps. If the procedure terminates, we have determined the maxi- 
mal safe set. The procedure is already described in full detail in [12,3]; here, we 
report only the definitions of the predecessor operators required to capture the 
previous notions. 

Discrete uncontrollahle predecessors operator Free ■ 

Pree{K) = {{q,x,^) £ Q x X x S : 

yuc £ a;,^) Bag £ x, such that 

(cTcCTe) yf (e, e) A S{q,x,^,ac,<Je) % K}. 

Discrete controllable predecessors operator Free ■ ^ 

Frec{K) = {{q,x,C> £ Q x X x S ■. 

3(Jc € a;,^) such thatyag £ a:, 

(fJcCTe) yf (e, e) A (5(g, a;, CTc, (Je) C K}. 

Continuous uncontrollable predecessor operator^ 

Unavoid.Fre : x 2('3><^X“): 

Unavoid-Fre{B, E) = {{q,x,^) G Q x AT x S' | 

\/u £ U 3t > 0 3d £ D such that, for the trajectory 
x{t) = %fq{u,d,x,ff){t) we have: {q,x{t,^)) £ B A 
Vr e [0,t) [u{t) £ Mf®(g,a;(r),^) A 
d(r) G Mf\q,x{T),C) A (q,x(r),^) £ Wait HE]} 



Figure 4 shows the fixed-point computation to obtain the maximal safe set. 
The procedure successively prunes away configurations that are found to lead to a 
bad configuration upon one additional discrete step (Pree(IT*)), or a continuous 
step to a bad configuration {Unavoid-Pre{Pree{W'‘) U IF*, Prec(IF*)). It is not 
guaranteed to stop in a finite number of steps. 

A hybrid controller watches the entire state of the system at all times, and 
decides whether to (1) take discrete control actions that may cause an instan- 
taneous change in the configuration, or to (2) let time pass under a continuous 
input u with the continuous variables evolving according to dynamics at the cur- 
rent mode. The formal definition of a safe hybrid controller and the rules which 
allow its extraction by the description of the maximal safe set are described 
in [4]. 

We define U = {«(.) G PC°\u{t) £ U,yt £ R.} and V = {d{.) £ PC°\d{t) £ 
D,yt £ ]R}, and we denote by Wait the set of configurations in which both players 
may choose not to play a discrete move, but instead wait for time to pass: Wait = 
{{q,x,^) I e G and e G x, ^)}. Trajectories at location q from 

initial state (x, C) following u £lA and d £T> are denoted tpq{u, d, x, ^). 
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W° ■- Good 
i := —1 
repeat { 

i := i + 1 

W*+-^ ■- \ [Pree{W^) U Unavoid.Pre{Pree{W^) gW, Pre^{W^))] 

} until = W^) 

Safe ■- W' 



Fig. 4. Computation of Maximal Safe Set [12]. 

5 Computation of the Maximal Safe Set for the Engine 
System 

The parameters of the hybrid model M of the engine running at idle have been 
identified by measurements provided by the Powertrain Division of Magneti 
Marelli on a commercial engine. 

In this section, for simplicity, we restrict the computation of the maximal safe 
set to the case in which only a positive spark advance is considered. In this case, 
the engine system is represented by the hybrid automaton M which consists of 
four discrete modes {S, S+, 

In M, there is a symmetry between the modes S, S+ (subsystem M 2 ) and 
the mirror modes (subsystem M^)- This allows the computation of the 

maximal safe set as follows: first, the maximal safe set is computed for the system 
M 2 representing the engine with the clutch open; then, using the previous results, 
the maximal safe set is derived for M^ representing the engine with the clutch 
closed; finally, the maximal safe set for the overall system M is obtained. 

The procedure reported in Section 4 is quite complex when applied to M 2 , 
due to the dimension of the continuous state space. However, since the set Good 
involves only the variable n, and variables (n, T, 9c) are de-coupled from the 
remaining ones, we can apply a divide-and-conquer strategy: the procedure is 
applied first in the subspace {n,T,6c); then, using (2), the safe values for the 
variable me are obtained in terms of n, T, 9c and 9s- Finally, from the safe 
values of the air mass, the safe values for the manifold pressure p are obtained, 
so that the overall maximal safe set Safe^^ for the hybrid subsystem M 2 is 
derived. For the engine parameters of our model, this computation terminates 
in six iterations. 

Due to symmetry, the maximal safe set for M^ is computed as the one for 
M 2 , yielding a result differing only in the coefficients of the crankshaft speed: 
when the clutch pedal is released the crankshaft inertial load increases, and 
On < an', < bn- Comparing the maximal safe sets for M 2 and M^, it holds: 



Safe^-\s C Safe^-\sL, 
•S'a/e“"|s+ C 5 ' 0 /e IsL. 



( 5 ) 
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(a) Modes S and S^, space (n, T, 9c) (b) Modes S and S^, space (p, n, 9c), 

with T = 15A^m and me = 480p(? 




me 

(c) Modes S+ and space 

(n, T, me), with 9c = 180° — 9s 
and 9s = 10° 




(d) Modes S and , space 
(n, T, me), with 9c = 0° 



Fig. 5. Maximal Safe Set for the Engine System M 



Finally, we consider the system M which contains the subsystems M 2 and 
together with the uncontrollable transitions S ^ , Sjc ^ S ^ and 

^ S+. In summary, one can prove the following: 

Proposition 1. If Safe^ is the maximal safe set for the engine system M , 



Safe^ls = Safe^lsL = Safe^^\s, 
Safe^\s^= Safe^\sL= Safe^^\s+. 



( 6 ) 



The maximal safe set for M is shown in Figure 5. Since the dimensions of the 
safe set are higher than three, it is difficult to visualize it. The three figures are 
projections of the safe set on different axes. The projection of the safe set on the 
subspace n,T,0c, shown in Figure 5(a) for modes S and , does not depend 
on the value of the other state components p,mc,ds- Note that the range of 
safe values for the speed n increases as 6c increases, whenever a given torque 
T is considered. This corresponds to the fact that the greater the values of 6c, 
the shorter is the interval of time on which the load torque may act to drive n 
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outside the good range [770,830], before the next dead point is reached and a 
new driving torque T may be applied. 

Figure 5(6) presents the dependencies among p, n, 9c, in modes S and S^, for 
given values of T and me- Note that the safe values for 9s are obtained by (2), 
which holds in S and S^. For a fixed value of 9c, there is an inverse dependence 
between the safe values of p and those of n. In fact, the greater the values of n, 
the smaller the values of p have to be in order to produce a small torque in the 
next expansion phase. 

For modes S+ and S^, safe set projections similar to those in Figure 5(a) 
and Figure 5(6) can be shown. 

The safe values for 9s are given by the relationship with n,T,nic, and 9c 
depicted in Figure 5(c). Note the inverse dependence between the safe values of 
me and the ones of n. The reason for this dependency is that, for high values 
of n, small values of me need to be applied in order to produce low values of 
torque T in the next expansion phase, so as to prevent the engine speed from 
exceeding the upper limit 830. 

Figure 5(d) shows, for each safe couple (n, T) at the beginning of the engine 
phase {9c = 0), the interval of values to which me must belong to produce a 
torque that maintains the speed engine n in the good range, provided that an 
appropriate value of 9s is chosen. 

The condition under which a non-empty maximal safe set exists for the engine 
hybrid model has been analytically determined in terms of the model parameters. 
By this result, the maximum value of the torque load for which there exists a non- 
empty maximal safe set was found to be 12.8 Nm for an engine of a commercially 
available vehicle. 

6 Conclusions 

The problem of maintaining the crankshaft speed within a given range has been 
formalized for the first time as a safety specification for the closed-loop system 
modeled as a hybrid automaton, where continuous and discrete variables retain 
their distinctive nature. By applying a systematic procedure to the hybrid model 
of the engine, the maximal safe set for the idle control has been determined. We 
also obtained as a by-product the entire set of possible controllers that satisfy 
the constraints. This, in addition to the capability of modeling in a separate 
but integrated manner discrete and continuous signals, is a definite advantage 
over other approaches that approximate the system by relaxing it to continuous 
or discrete sampled representations. This result is the first of its kind in idle 
control, and allows us to determine tightly the maximum range of allowed torque 
disturbances, given the maximum interval of angular speed values. Further, from 
an application point of view, the relevance of this computation lies in the fact 
that it provides the upper bounds on the best performance achievable by an idle 
speed control strategy for a given engine. 

The systematic procedure for the safe set computation cannot be guaranteed 
to converge in a finite number of steps in the general case. However, in the engine 
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model available to us it converged in six steps. We are presently investigating 
the properties of the model and the range of parameters with respect to the 
size and the shape of the the corresponding maximal safe set for the idle engine 
control problem. 
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Abstract. In this paper, we formulate the problem of characterizing the 
stability of a piecewise affine (PWA) system as a verification problem. 
The basic idea is to take the whole IR" as the set of initial conditions, 
and check that all the trajectories go to the origin. More precisely, we 
test for semi-global stability by restricting the set of initial conditions 
to an (arbitrarily large) bounded set T’(O), and label as “asymptotically 
stable in T steps” the trajectories that enter an invariant set around the 
origin within a finite time T, or as “unstable in T steps” the trajectories 
which enter a set X-mst of (very large) states. Subsets of X{0) leading 
to none of the two previous cases are labeled as “non-classifiable in T 
steps”. The domain of asymptotical stability in T steps is a subset of 
the domain of attraction of an equilibrium point, and has the practical 
meaning of collecting the initial conditions from which the settling time 
to a specified set around the origin is smaller than T. In addition, it can 
be computed algorithmically in finite time. Such an algorithm requires 
the computation of reach sets, in a similar fashion as what has been 
proposed for verification of hybrid systems. In this paper we present a 
substantial extension of the verification algorithm presented in [6] for 
stability characterization of PWA systems, based on linear and mixed- 
integer linear programming. As a result, given a set of initial conditions 
we are able to determine its partition into subsets of trajectories which 
are asymptotically stable, or unstable, or non-classifiable in T steps. 



1 Introduction 

Hybrid models describe processes which evolve according to dynamics and logic 
rules. Hybrid systems have recently grown in interest not only for being theo- 
retically challenging [10], but also for their impact on applications, for instance 
in the automotive industry [3]. 

* Corresponding author. 
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An important class of hybrid systems are the so-called Piecewise Affine 
(PWA) systems. These are defined by partitioning the state-space into poly- 
hedral regions, and associating with each region a different linear state-update 
equation. PWA systems can model a large number of physical processes, such 
as systems with static nonlinearities (for instance actuator saturation), and can 
approximate nonlinear dynamics with arbitrary accuracy via multiple lineariza- 
tions at different operating points. The study of PWA systems is also motivated 
by the stability and performance analysis of high-performance controllers [20]. 
In particular, recently in [7] the authors show that a model predictive controller 
(MPC) for constrained linear systems can be explicitly expressed in closed-form 
as a continuous and piecewise affine state-feedback law. The resulting closed-loop 
system is therefore PWA, and criteria for proving stability and robust stability 
against disturbances and model uncertainties are of fundamental importance. 

PWA systems are equivalent to interconnections of linear systems and finite 
automata, as pointed out by Sontag [26]. Based on different arguments, a sim- 
ilar result was proved constructively in [4], where the authors show that PWA 
systems are equivalent to the hybrid mixed logical dynamical (MLD) systems 
introduced in [5]. MLD systems are capable to model a broad class of systems 
arising in many applications: linear hybrid dynamical systems, hybrid automata, 
nonlinear dynamic systems where the nonlinearity can be approximated by a 
piecewise linear function, some classes of discrete event systems, linear systems 
with constraints, etc. Examples of real-world applications that can be naturally 
modeled within the MLD framework are reported in [5, 6] . The MLD framework 
allows specifying linear dynamics x' = Ax + Bu, any logic proposition, and the 
interaction between the two. The key idea of the approach consists of embedding 
the logic part in the state equations by transforming Boolean variables into 0-1 
integers, and by expressing the relations as mixed-integer linear inequalities [5]. 

Despite the fact that PWA systems are just a simple extension of linear 
systems, they can exhibit very complex behaviors, as typical of nonlinear sys- 
tems [24]. Blondel and Tsitsiklis [9] showed that even in the simple case of 
two component subsystems, verifying the stability of autonomous discrete-time 
PWA systems is either an AfP-hard problem (no polynomial-time algorithm), 
or undecidable. In view of these complexity results, no hope remains of finding 
criteria for stability of PWA systems as easy as for instance the Routh-Hurwitz 
rule for linear systems. Stability of each linear subsystem is not enough to guar- 
antee stability of the overall system (and vice versa) [11, 28], as the switching 
rule between linear dynamics is fundamental for stability of the interconnection. 
Some criteria for stability of PWA systems were recently proposed, which are 
based on piecewise quadratic Lyapunov functions computed by solving linear 
matrix inequalities (LMI) [16], and multiple Lyapunov functions methods [11]. 
However, LMI based approaches have the drawback of being conservative, the 
more conservative the larger the number of regions in the polyhedral partition 
of the state space. 

Complexity results were also shown in [4] for AfP-completeness of observabil- 
ity analysis, and undecidability of reachability in the context of formal verifica- 
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tion of hybrid automata is well known [1, 18]. The problem of formal verification 
can be simply stated as follows: For a given set of initial conditions and distur- 
bances, certify that all possible trajectories never enter a set of unsafe states, 
or possibly provide a counterexample. In spite of this complexity, several tools 
for formal verification of hybrid systems have been proposed in the literature, 
mainly for linear hybrid automata [15, 19]. 

In this paper, we formulate the problem of characterizing the stability of a 
PWA system as a verification problem. The basic idea is to check for reachability 
from an (arbitrarily large) bounded set T(0) of initial conditions to (i) a set 
around the origin, and (ii) a set of very large (=unsafe) states. More precisely, 
we label as “asymptotically stable in T steps” the trajectories that enter an 
invariant set around the origin within a finite time T, or as “unstable in T 
steps” the trajectories which enter a (very large) set Tinsf Subsets of X{0) 
leading to neither of the two previous cases are non-classified. Such a verification 
problem of “practical” stability is decidable. Many undecidable problems can be 
approximated by decidable ones which are equivalent from a practical point of 
view. The decidable algorithm shown in [4] for analysis of observability is another 
example of such a philosophy. 

In order to solve the problem of verification of stability, we substantially 
extend the algorithm proposed in [6]. Safety tests and reach set computation 
are done via linear programming (LP), switching detection via mixed-integer 
linear programming (MILP), and approximation of the reach set by using tools 
from computational geometry. In particular, with respect to [6], we make the 
algorithm more efficient, and use an algorithm for arbitrarily precise inner and 
outer approximation of polyhedra [8] . 

The approach followed in this paper is related to the idea of robust simu- 
lation [17], which consist of simulating entire set evolutions rather than single 
trajectories for stability and performance analysis. In [17] the author tests for 
finite time stability by computing an outer approximation of the reach set via 
mathematical programming. However, an outer approximation is performed at 
each time step in order to bound the complexity of the reach set. It turns out 
that the approach provides only a sufficient condition to conclude about the 
stability of the initial set. On the contrary, in this paper an exact characteri- 
zation of the initial set is obtained by first applying a verification algorithm to 
the system, and then by refining the results through linear programming. By 
removing all conservativeness, this allows partitioning the initial set into three 
subsets: (i) states belonging to the domain of asymptotic stability in T steps, {ii) 
states belonging to the domain of instability in T steps, and (in) states which 
are non-classifiable in T steps. 

2 Hybrid and Piecewise AfRne Models 

Several modeling frameworks were proposed in the literature. Two main cat- 
egories were successfully adopted for analysis and synthesis purposes [10]: hy- 
brid control systems [1, 2, 5, 21, 22], which consist of the interaction between 
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continuous dynamical systems and discrete/logic automata, and switched sys- 
tems [11, 16, 25], where the state-space is partitioned into regions, each one 
being associated to a different continuous dynamics. 

Switched systems defined by a polyhedral partition of the state-space and 
linear dynamic equations are the so-called piecewise affine (PWA) systems 

x{t -I- 1) = Aix{t) Biu{t) fi , for x{t) e Ci = {x : H^x < Ki} (1) 

where x G X C ]R”, u G M™, {Ci}®rg is a polyhedral partition of the sets of 
states X, and ft is a constant vector. A trajectory is the collection of vectors 
{a:(0),... ,x{t),...} satisfying the difference equation (1). Without additional 
hypotheses on continuity of the piecewise affine state-update mapping, defini- 
tion (1) is not well posed in general, as the state-update function is twice (or 
more times) defined over common boundaries of sets Ci (the boundaries will be 
also referred to as guardlines) . This is a technical issue which can be avoided as 
in [25]. 

In [4] the authors show that PWA systems are equivalent to the mixed logic 
dynamical (MLD) systems introduced in [5]. These are hybrid (control) systems 
defined by the interaction of logic, finite state machines, and linear discrete-time 
systems, defined by the equations 

x{t -I- 1) = Ax{t) H- B\u{t) 826(1) -4- B^z{t) (2a) 

£26(1) -\- £3z(t) < £iu(t) £ix(t) £5 (2b) 

where x G IR"° x {0, 1}”^ is a vector of continuous and binary states, u G IR'"° x 
{0, 1}'"^ are the inputs, and 6 G {0, 1}’’% z G represent auxiliary binary and 
continuous variables respectively, which are introduced when transforming logic 
relations into mixed-integer linear inequalities [23, 27], and A, Bi, B2, B3, £\, 

. . . ,£3 are matrices of suitable dimensions. Throughout the paper, we will assume 
that both the PWA and the MLD forms are available. Their complementary role 
in the verification algorithm will be discussed later. 



3 Stability Characterization Problem 



As mentioned in the introduction, determining the stability of PWA systems can 
be a complex task. Nevertheless, we aim at estimating the domains of attraction 
of equilibrium points, and the set of initial conditions from which the state 
trajectory reaches magnitudes greater than an arbitrarily large value. 

For simplicity of exposition, from now on we will assume that the system is 
piecewise linear (fi = 0, for all z = 0, . . . , s — 1), and autonomous (Bi = 0 for all 
z = 0,... ,s — 1)^, and that the only equilibrium point (the origin) belongs to the 



^ Robust stability questions in the presence of disturbances u(t) G ht, where W is a 
given bounded set, can be similarly formulated. 
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interior of one of the sets of the partition^ , which by convention will be referred 
to as Cq. Denote by I?oo(0) C ]R" the (unknown) domain of attraction of the 
origin (if the origin is unstable then I?oo(0) = {0}). Given an (arbitrarily large) 
bounded set A’(O) of initial conditions, we want to characterize I?oo(0) p| A’(O). 

A necessary condition for the origin to be asymptotically stable is that the 
matrix Aq associated with the region Cq is strictly Hurwitz. Under this assump- 
tion, we can compute an invariant set in Cq. In particular, we compute the 
maximum output admissible set (MOAS) Aoo C Cq. Xao is the largest invariant 
set contained in Cq, which by [14, Th.4.1] is a polyhedron with a finite number of 
facets, and is computed through a finite number of linear programs (LP’s) [14]^. 

In order to circumvent the undecidability of stability mentioned above, we 
define the following 

Definition 1. Consider the PWA system (1), and let the origin 0 € Cq = 
{x : Hqx < Kq}, and Aq be strictly Hurwitz. Let be the maximum output 
admissible set (MOAS) in Cq, which is an invariant for the linear system x{t + 
1) = Aox(t). Let T be a finite time horizon. Then, the set A(0) C IR” of initial 
conditions is said to belong to the domain of attraction in T steps I?t(0) of the 
origin if\/x{0) € X{Q) the corresponding final state x(T) S X^o. 

Note that Vt(0) C Vt+i(0) C I?oo( 0), and Vt(0) — > I?oo(0) as T ^ oo. The 
horizon T is a practical information about the speed of convergence of the PWA 
system to the origin. 

Definition 2. Consider the PWA system (1), and let Xinst C IR” The set 
X{Q) C IR" of initial conditions is said to belong to the domain of instability in 
T steps Xt(0) if'ix{0) G X{0) there exists t, 0 < t < T such that x{t) G Ai„sf 

In Definition (2), the set Xinst must be interpreted as a set of “very large” 
states. Although instability in T steps does not guarantee instability (for any 
finite T, a trajectory might reach Ainst and converge back to the origin), it has 
the practical meaning of labeling as “unstable” the trajectories whose magnitude 
is unacceptable, for instance because the PWA system is no longer valid as a 
model of the real system. Instability in T steps represents a condition of loss of 
safety for the PWA system. 

As I?t( 0) and It( 0) can have a nonempty intersection, we introduce the 
following 

^ The hypothesis of having equilibria only in the interiors of sets Ci, although restric- 
tive, is certainly satisfied when (1) is the result of the linearization of a nonlinear sys- 
tem around different equilibria, and is needed later for easily computing nonempty 
invariant sets. Moreover, the approach of this paper can be straightforwardly ex- 
tended to handle multiple equilibria of the PWA system which are not on the border 
of the polyhedral partition. These can be easily detected by standard linear analysis, 
and a maximum output admissible sets can be computed for each equilibrium. 

® If the effect of perturbations u{t) G W C IR’”, where W is a given bounded set of 
disturbances and Bq A 0, has to be taken into account X^o is the largest invariant 
set under disturbance excitation, and can be computed as proposed in [13]. 
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Definition 3. Consider the PWA system (1). The set X{0) C ]R" of initial 
conditions is said to belong to the domain of safe stability in T steps 5 t(0) if 
5t(0) C Vt{0) and 5t(0) = 0 . 

Definition 3 describes trajectories which asymptotically converge to the origin 
without crossing the set X-mst- 

Given a set of initial conditions X{Q), we aim at finding subsets of X{Q) 
which are safely asymptotically stable (df(0) p| 5 t( 0)), and subsets which lead 
to practical instability in T steps (df(0) P|Xt( 0)). Subsets of X{0) leading to 
none of the two previous cases are labeled as non- classifiable in T steps As 
we will use linear optimization tools, we assume that X{Q) and IR"\A’inst are 
convex polyhedral sets. Typically, non-classifiable subsets shrink and eventually 
disappear for increasing T . 

3.1 Switching Sequences 

The evolution of the PWA system (1) for u{f) = 0, /i = 0, Vt = 0, . . . , s — 1, is 
given by 

x{t) = ■ • • Ai(o)a:(0) (3) 

where in (3) i{k) G {0,... ,s — 1} is the index such that Hi(^!^)x{k) < 
k = 0,. . . , t — 1, is satisfied. The previous questions of practical stability can be 
answered once all the switching sequences I{t) = {z(0), ... ,t(t — 1)} leading to 
Xao or Xinst from A(0) are known. In fact, for safe stability in T steps it is enough 
to check that the reach set at time T, X{T,X{0)) = Ai(^x-i)^i(T- 2 ) ' ' ' ^i(o)'^(0)> 
satisfies the set inclusion X{T, A(0)) C X^o for all admissible switching sequences 
I{T). However, the number of all possible switching sequences I{T) is combinato- 
rial with respect to T and s, and any enumeration method would be impractical. 
In the next section we show that a verification algorithm can be used to avoid 
such an enumeration. 



4 Verification 

In order to determine admissible switching sequences I(t), we need to exploit 
the special structure of PWA systems (1). This allows an easy computation 
of the reach set, as long as the evolution remains within a single region Ci. 
Whenever the reach set crosses a guardline and enters a new region Cj, a new 
reach set computation based on the j-th linear dynamics is computed, as shown 
in Fig. 1(a). 

Let X{0) be a convex polyhedral set, and partition it into subregions 4^(0) = 
X{0)f^Ci, z = 0, ... ,s — 1. For all nonempty sets 4^(0), computing the evo- 
lution X(T, Xi(0)) requires: (z) the reach set A(t, 4^(0)) p| Ci, i.e. the set of 
evolutions at time t in Ci from Ai(0); (zz) crossing detection of the guardlines 
Vh — X{t,Xi{0))f]Ch yf 0 , V/z = 0, . . . , z — 1, z -I- 1, . . . , s — 1; (zzz) elimination 
of redundant constraints and approximation of the polyhedral representation of 
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MO) 



(a) Reach set evolution, guardline 
crossing, outer approximation of a new 
intersection 



(b) Outer rectangular approxi- 
mation of a polytope 



Fig. 1. Reachability Analisys 



the new regions Vh (approximation is desirable, as the number of facets of Vh 
can grow linearly with time); {iv) detection of emptiness of X(t,Vh) (empti- 
ness happens when all the evolutions have crossed the guardlines), detection of 
safe stability X{t,Vh) C X^o, detection of practical instability X{t,Vh) C Xi^st 
(these three will be referred to as fathoming conditions). 

4.1 Reach Set Computation 

Let the set of initial conditions be defined by the polyhedral representation 
fb(0) = {a;: Sqx < Tq}. The subset S of X{0) whose evolution lies in Ci for t 
steps is given by 



As S' is a polyhedral set, the reach set X{t, Xi{0)) f\Ci = A(S is a polyhedral set 
as well. In the presence of input disturbances and nonzero offsets /i, S = {a; G 



which is a polyhedron in the augmented space of tuples {x, w(0), . . . , u{t— 1)). A 
compact representation of the set Xft, Xi{Q)) p|Ci (as inequalities over the final 
state x{t)) can be computed by a geometric projection procedure, for which 
efficient tools exist, e.g. [12]. 

4.2 Guardline Crossing Detection 

Switching detection amounts to finding all possible new regions C^’s entered by 
the reach set at the next time step, i.e. nonempty sets Vh — df(t, Ai(0)) p| C/i, 
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Fig. 2. Graph of evolution G 



h ^ i. Rather then enumerating and checking nonemptiness for all /i = 0, . . . — 

1, i + 1, . . . , s — 1, we can exploit the equivalence between PWA systems and 
MLD models (2), and solve the switching detection problem via mixed-integer 
linear programming. More in detail, in the MLD form the condition x{t) € Ch is 
associated to the condition 5{t) = Sh & {0, 1}’’^, for instance x{t) € C 5 4^ S(t) = 
[1 0 1]'. Switching detection amounts to finding all feasible vectors S(t) G {0,1}’'^ 
which are compatible with the constraints in (2) plus the constraint x(t — 1) G 
X{t — 1, Ai(0)) O Ci- Such a problem is a mixed-integer linear feasibility test 
(MILFT), and can be efficiently solved through standard recursive branch and 
bound procedures. Thus, in the average case the MLD form (through the branch 
and bound algorithm) requires only a very small number of feasibility tests, while 
the PWA form would require for enumerating and solving a feasibility test for 
all the possible regions. 



4.3 Approximation of Intersection 

The computation of the reach set proceeds in each region Ch from each new 
intersection Vh- A new reach set computation is started from Vh, unless Vh is 
contained in some larger subset of Ch which has already been explored. As in 
principle the number of facets of Vh grows linearly with time, we need to approx- 
imate Vh so that its complexity is bounded (and therefore reach set computation 
from Vh has a limited complexity with respect to the initial region), and check- 
ing for set inclusion is a simple task. Hyper-rectangular approximations are the 
best candidates, as set inclusion between hyper-rectangles reduces to a simple 
comparison of the coordinates of the vertices. On the other hand, a crude rectan- 
gular outer approximation of Vh can lead to explore large regions which are not 
reachable from the initial set A’(O), as they are just introduced by the approxi- 
mation itself. In [8] the authors propose an iterative method for inner and outer 
approximation which is based on linear programming, and approximates with 
arbitrary precision polytopes by a collection of hyper-rectangles, as depicted in 
Fig. 1(b). 
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Fig. 3. Adding and removing nodes to the graph G 



4.4 Fathoming 

In Sect. 4.1 we showed how to compute the evolution of the reach set X{t,Vh) 
inside a region Ct. The computation is stopped once one of the following happens: 

1. The set X{t,Vh) C\Ci is empty. This means that the whole evolution has left 
region Ci. 

2. X{t,Vh) Q -Too, i-e. all possible evolutions from Vh are safely stable. 

3. X{t,Vh) Q Tinst, i-e. all possible evolutions from Vh have violated the con- 
dition for safe stability. 

4. The time t > T. 

These conditions can be checked through linear programming. 

4.5 Graph of Evolution 

The result of the exploration algorithm detailed in the previous sections can 
be conveniently represented on a graph G (Fig. 2). The nodes of G represent 
sets from which a reach set evolution is computed, and an oriented arc of G 
connects two nodes if a transition exists between the two correspoding sets. 
Each arc has an associated weight which represents the time-steps needed for 
the transition. The graph has initially no arc, and nonempty initial sets Xi(0) and 
dfoo, Tinst as nodes. As long as a new intersection X{t, Xi{0)) f^Ch is detected, 
it is approximated by a collection of hyper-rectangles, as described in Sect. 4.3. 
Each hyper-rectangle becomes a new node in G, and is connected by a weighted 
arc from Xiifi). In addition, each hyper-rectangle is pushed on a stack of sets to 
be explored. 

Before starting a new reach set computation from a set Rj extracted from 
the stack, we check for inclusion of Rj in other nodes of G. If this happens, say 
Rj C Ri and Rj C i ?2 as in Fig. 3, the node associated with Rj is removed from 
G, and all arcs pointing to Rj are directed to both Ri and i ?2 (dotted arrows). 
Finally, whenever the reach set hits dfoo (or Tinst), an arc is drawn from Vh to 
Too (or Tinst)- 
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After the verification algorithm terminates, the oriented paths on G from 
initial nodes Xi{Q) to terminal nodes X^o and Xmst determine a superset of 
feasible switching sequences I{t) = {f(0), ... — 1)}. In fact, because of the 

outer approximation of new intersections Vh, not all switching sequences are 
feasible. Nevertheless, feasibility can be simply tested via linear programming. 
Once all feasible switching sequences I{t) have been identified, the partition of 
the initial set into safely stable and unstable regions is determined by the sets 
■ ■ •^i(O)'^(0)) t <T. 

Algorithm 1. 

1 initialize GRAPH with nonempty initial nodes Xi{0), z = 0, . . . , no, 
and disjoint final nodes !Fj, j = 1, . . . ,nf, 

2 push in STACK Aj(0), z = 0, ..., no; 

3 while STACK nonempty do 

4 pop region Rj from STACK, and let z such that Rj C Cf, 

5 if no region in GRAPH includes Rj then 

6 t ^ t* = minimum arrival time from initial nodes to Rj\ 

7 for j = 1, . . . ,Uf do 

8 if X(t,Rj) C Tj then go to 20 \ 

9 if X{t, Rj) CMFj then 

10 connect Rj to Tj with weight t — t*\ 

11 t ^ t+ 1 \ 

12 X{t,R,) = A,X{t - l,Rj) + BiU + {fi\\ 

13 for all fz yf z such that Vh — Ch p| A’(t, Rj) yf 0 do 

14 insert Vh in GRAPH and connect Rj to Vh with weight t—t*', 

15 push Vh on STACK; 

16 X{t,Rj) ^ X{t,Rj)f]Cj-, 

17 if X(t,Rj) yf 0 and t <T then go to 9 ; 

18 else 

19 redirect all arcs to Rj to all regions Rh in GRAPH, Rh V Rj\ 

20 end . 



4.6 Verification Algorithm 

The techniques proposed in the previous sections for verification of PWA systems 
are summarized in Algorithm 1. In step 1 , Vi = X^o and T 2 = Tinst- Step 6 is 
computed by standard techniques for shortest path computation, while step 13 
by branch and bound. In step 14 , the collection of hyper-rectangles computed by 
outer approximating Vh are put on the stack, rather than Vh- 

Note that Algorithm 1 can be generalized to verification purposes, by inter- 
preting T\ as a set of target states, and T-i as a set of unsafe states. Moreover, 
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Fig. 4. PWA system (5), initial region V(0), MOAS Xoc, and trajectories of the system 



linear programs can be performed during reach set computation in order to de- 
termine the range of given state components. The algorithm can be extended to 
include disturbances u(t) G where U is & given bounded polyhedral set, at 
the price of more complicate computations (see footnote 3). 

We finally remark that the termination of Algorithm 1 after a finite time is 
guaranteed because no exploration is performed for t > T (step it). 



5 An Example 



Consider the PWA system 
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and let <T(0) = {x € : ||x||oo < 2}, Ainst = {x G : ||x||oo > 10}. The 

origin is asymptotically stable, as Aq has eigenvalues 1 ± j 1 . The corresponding 
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(a) r = 5 (b) T = 12 

Fig. 5. Stability characterization of system (5) 



was computed by the algorithm in [14] . A simulation of the system from different 
initial conditions is depicted in Fig. 4, which shows that the trajectories either 
converge to the origin or diverge to infinity. We characterize the set of initial 
conditions by running Algorithm 1. The results are shown in Fig. 5. With the 
time horizon T = 5, not all the set of initial conditions is classified for stability 
(the darkest subsets are non-classifiable in 5 steps). By augmenting the time 
horizon, the region of states which are non-classifiable in T steps shrinks, and 
disappears for T = 12. Algorithm 1 is implemented in Matlab 5.3 on a Pentium 
II 400, and requires 57 s to produce the plot in Fig. 5(b) (T = 12). 
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Abstract. A structural procedure is proposed for solving the problem 
of maximal safe-set determination based on maximal controlled invari- 
ant sets. However, the procedure is not guaranteed to converge in a finite 
number of steps. The procedure is made computationally appealing first 
by linearizing and discretizing the dynamical systems and, second, by 
using an inner approximation of these sets that, together with the clas- 
sical outer approximation, yields tight bounds for an error due to the 
truncation of the procedure after a finite number of steps. The theory is 
applied to idle-speed regulation in engine control. 

1 Introduction 

Hybrid systems have been the subject of intensive study in the past few years. 
In particular, emphasis has been placed on solving problems with safety spec- 
ifications, which are described by giving a set of good states within which the 
controlled hybrid system should evolve. The set of all initial states guaranteeing 
that the evolution of the system remains in the good set is the maximal con- 
trolled invariant set contained in the set of good hybrid states. This set is called 
maximal safe set and the set of all control strategies which make this set invari- 
ant is the maximal controller. A systematic procedure for solving problems with 
safety specifications has been proposed in [21], [18]. The procedure is not, how- 
ever, guaranteed to converge in a finite number of steps and is computationally 
complex. 

In [4] and [5] we analyzed this problem for a restricted class of hybrid systems, 
called switching systems, with the goal of obtaining computationally efficient 
procedures. Switching systems are characterized by a finite state machine (FSM) 
and a set of dynamical systems, each corresponding to a state of the FSM. The 
transitions between two different states of the FSM are determined by external 
uncontrollable events which act as discrete disturbances. The motivation to study 
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this class of systems came from the application of hybrid systems techniques to 
the automotive engine control problem [2] . An algorithm for the determination of 
the safe set was proposed that presents an important computational advantage 
over the general procedure of [21], [18] obtained by exploiting the structure of the 
FSM. The problem was decomposed into a number of different sub-problems that 
consist of finding a robust controlled invariant set for a given dynamical system. 
The theory was applied to idle-speed regulation for automotive engine control. 
To do so, we gave a procedure that follows the essential ideas formalized in this 
paper: we linearized and sampled the nonlinear dynamical system describing the 
engine behavior and we remarked that these ideas have general applicability. 

In this paper, we show how our procedure can be generalized to the case 
of a general hybrid system. Since no general procedure is known for the deter- 
mination of maximal controlled invariant sets for nonlinear dynamical systems, 
we propose to linearize and use a discrete-time representation of the nonlin- 
ear dynamical systems as an important step towards a computationally efficient 
approach. In fact, for discrete-time linear systems and polyhedral constraining 
sets, several results for the computation of maximal controlled invariant sets have 
been reported in the literature (see e.g. [6], [11], [12], [15], [17]). We then propose 
numerical methods for the computation of controlled invariant sets for discrete- 
time linear systems and polyhedral constraining sets based on the results of [9] . 
Even in this simpler case, the procedure for the computation of the maximal 
controlled invariant set may not converge in a finite number of steps. Hence, 
we propose a procedure for approximating the maximal controlled invariant set 
and we show how to obtain an accurate bound of the error by combining inner 
and outer approximations. We then proceed to show how to choose a discretiza- 
tion in order to obtain a precise relation between the invariant sets associated 
with general continuous-time dynamical systems and those of the corresponding 
discrete-time systems. Finally, we solve the idle control problem for automotive 
engines using our approach. 



2 Switching Systems 

Switching systems can be considered indexed collections of dynamical systems, 
each determining the evolution of the system, except during those instants of 
time in which there is a “jump” between two different dynamical systems. This 
jump is uniquely determined by external events, which act as discrete distur- 
bances. Switching systems can be defined following the general model of hybrid 
automata given in [18]. 

Definition 1. (Switching Systems) A switching system is a tuple: 

n = (Q, A, U, Y, Sc, s, E, R), (or, respectively, U = (Q, A, U, Y, So, S, E, R) ) 

where: 

~ Q is a finite collection of discrete state variables taking values in the set of 
discrete states Q — {q\,q 2 , 
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— X is a finite collection of continuous state variables taking values in the 
continuous state space X =R"; 

— U is a finite collection of input variables. We assume U = Ud ^ Uc U Ud 
where Ud contains discrete and Uc U Ud contains continuous variables. 
Variables inUo take values in the set Uo and they are regarded as discrete 
disturbance variables. Ud contains a special element which we denote by e. 
Variables in Uc (resp. Ud) take values in the set Uc = R™ (resp. Ud C M’’ 
) and they are regarded as control (resp. disturbance) variables. Moreover we 
denote by Uc the class of (continuous) control functions and by Ud the class 
of continuous disturbance functions. 

— Y is a finite collection of continuous output variables, taking values in the 
set Y = W. 

— Sc is the class of continuous time dynamical systems defined by the equa- 
tions: 

x{t) = fi{x{t),u{t),5{t)) .. 

y(t) = h,{x{t),u{t)) 

where t G M., J = 1, ..., N CN , and fi is such that, Vm (•) G Uc , V<5 (•) G Ud, 
the solution x (t) of each differential equation, for i G J, exists and is unique, 
(or, respectively, Sd is the class of discrete time dynamical systems defined 
by the equations: 

x{t-Gl) = fi {x (t) ,u{t),5 (t)) , . 

y{t) = h,{x{t),u{t)) 

where t G Z, J = 1, ..., N cN , and fi is such that, Vu (•) G Uc , V(5 (•) G Ud, 
the solution x (t) of each difference equation, fori G J , exists and is unique). 

— 5 : Q — > Sc (or, respectively, S : Q ^ Sd) is a mapping associating 
to each discrete state of the switching system a continuous time ( or, respec- 
tively, a discrete time) dynamical system. 

— EcQ X Ud X Q is a collection of discrete transitions; E is such that: 1 ) 
{q, e,q) G E , Vq G Q; 2) if {q, e, q') G E then q = q' ; 3) if {q, a, q') G E, 
a ^ e, then q ^ q' . 

— R : E X X. ^ ~K assigns to each {q, a, q') G E a reset function. 

The triple (Q,Ud,E) can be viewed as an FSM having state set Q, inputs 
Ud (external events) and transitions defined by E. In our case, which event in 
the set E determines a switching doesn’t play a direct role. We call the pair 
{q, S (q)) consisting of the discrete state q and the associated dynamical sys- 
tem S (q) a switching system configuration. 

We now define the evolution in time of a switching system. First, following 
[18], we introduce the concept of hybrid time basis for the temporal evolution of 
switching systems. Let us denote by T the set M’*', or respectively the set Z+, 
depending on whether we are considering switching systems with 5 : Q ^ Sc 
or: S : Q ^ Sd . 

Definition 2. (Hybrid Time Basis) [18] A hybrid time basis t is a finite or 
infinite sequence of sets R , i Gf^ satisfying the following conditions: 
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— li is of the form li = {t € T : U < t < tf} unless t is a finite sequence and 
II is the last set of the sequence, in which case it can he of the form II = 
{t€T:t>tL} ; 

— For all i, U < t[ and for i > 0, U = t[_-^ . 

We denote by T the set of all hybrid time bases. We define now an execution 
of a switching system, which describes its evolution in time. 

Definition 3. (Switching System Execution) Let a function tp : Q ^ T be 
given. An execution x of a switching system hi is a collection x= {t, q, x, a, u, 6, y) 
with T G T , q : T ^ Q, a;:r— >X, <T:r^UD,u:r— > Uc with u (•) G Uc 

S : T ^ Ud with S (•) G Ud, and y : t — > Y satisfying: 

— Minimum permanence time in each configuration: r = {li} is such that: 
t'i -U> ip{q{ti)). 

— Continuous evolution: For all i with F < t'^ : 

• X, u, S, y are continuous and q is constant over li . 

• Vt G h, X (t) is the (unique) solution of S {q{ti)) with initial condition 
x{ti), given some control input u(-) and some continuous disturbance 
6 (•), and y (f) is the corresponding output. 

— Discrete evolution: For all i, Ci = {q (t [) , cr (t') , q (ti+i)) G E and x (ti+i) = 
R{e„x{ti)) . 



Problem 1. Consider the switching system of Definition 1. For i = 1,...,N, let 
L2{qi) C RP be a given set associated with state qt G Q. Let go be an element 
of Q. Find the set Xg C M” of all possible continuous initial states such that 
3m(-) G Uc such that for any execution with q (to) = qo, x (to) G Xq, u(-) = u(-), 
the following constraints are satisfied, Vi5 (•) G Ud'. 

y(t) G f2(q(t)), Vt > to (3) 

In what follows, we set for simplicity, and w.l.o.g., L2{qi) = fi, Vz = 1, ..., N . 
Moreover, for the sake of notational simplicity, we set At = (p (qf), qi G Q. 

Remark 1. It is possible to use the formulation of Problem 1 to deal with ap- 
proximate output tracking problems. Given a reference trajectory yR(t) for the 
output, we require that the output of our switching system at any instant of 
time, differs from the given reference for at most a prescribed quantity e. If the 
reference trajectory is the output of an exosystem: 

(w = s{w) 

1 ynit) = q{w), w(o) = wo ^ ’ 

(where, for simplicity, only the case of continuous time systems has been consid- 
ered), incorporate (4) into the model of the switching system and define a new 
output function as yE{t)=y{t)—yn{t). Moreover, set C = {yE : —si < yE < £1} ■ 
Then, the approximate tracking problem is formulated in the form of Problem 1. 
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3 Problem Solution 



To state the main results, the following definitions are needed. 

Definition 4. A set S C K" is robustly controlled invariant with respect to 
configurations {{q,S (q)) , q € Q ' C Q } and constraints (3) if: Vg G Q', 'ix G 
E, 3m (•) G Uc such that the solution x (t) of S (q) with x (to) = x is such that 
x{t) € E and y (t) G 17, Vt > to, WS (•) G Ud ■ 

Definition 5. Given some set A C M”, define 

r a; G M" : 3m (•) G Uc such that y (t) G 17, 1 
f2^ (yl) = < for all to <t <to + ‘A and > 

X {to + A) G yl, V(5 (•) G Ud J 

where y{t) is the output of S {qi) with x {to) = x. 

Let Ii{A) he the maximal robust controlled invariant set with respect to con- 
figuration {qi,S {qi)) and constraint (3) contained in the set A. 



Definition 6. Given some set A C M", define 



(^) = 



a; G K” : 3cr G Ud such that 
R{{qi,a,qj) ,x) G yl 



If R{{qi,a,qj) ,x)=x for all {qi,<J,qj) G E, we say that R is the identity 
reset function. 



A connected FSM can be decomposed into its strongly connected components 
(maximal sets of mutually reachable states) Fi,F 2 , - ■ ■ , Fm and there is a partial 
ordering among the strongly connected components. The strongly connected 
components of F determine a Directed Acyclic Graph (DAG), T, where the 
nodes correspond to Fi, T 2 , • • • , Fm- Without loss of generality, we assume that 
the DAG is rooted, e.g., there is only one node that has no incoming arc. 



Definition 7. A node qi G Q is closed if the set that solves Problem 1 with 
q {to) = qi, denoted Ei, has been found. Otherwise, qi G Q is open. 

Algorithm 1: Structural algorithm for the determination of the safe 
set 

MAIN: 

Init: Set Aj = 0, f = 1, ..., A. Let A be the set of nodes belonging to strongly 
connected components containing only one node. Let M be the set of strongly 
connected components containing two or more nodes. 

Repeat 

Do Find an open node v = qi G A such that either it has no successors or 
all its successors are closed. Solve the node v applying procedure Star{ v ) and 
mark it as closed. 

While no such node could be found. 
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Do Find a strongly connected component F € M such that each node be- 
longing to F either has no successors not belonging to F or all its successors 
not belonging to F are closed. Let Ip = {gi, < 72 , •••, 9Afp} C Q be the set of 
nodes belonging to F. If R restricted to F is the identity reset function and 
Ai = 0 for i = l,...,Np, solve the strongly connected component F by apply- 
ing procedure Strongly.Connected.Simp{F) and mark all its nodes as closed. 
Otherwise, solve the strongly connected component F by applying procedure 
Strongly.Connected{F) and mark all its nodes as closed. 

While no such strongly connected component could he found. 

Until the root node has been marked as closed. 

SUB Star{ v ) If v = qt has no successors, set Si = Xi (M”). Otherwise, let 
Isucc = {qy, C Q be the set of nodes that are successors of the node v. 

Let E, = (l, If = 0 then EXIT. 

SUB Strongly.Connected( F ) 

For all qi G Ip, let Ipsucc = {fti, Qimi} C Q be the set of nodes that are 
successors of the node qi. In general, the set Ipsucc contains nodes not belonging 
to F . If <7r G Ipsucc and qr ^ Ip then, in what follows, we let E^ = Sr, Vfc, the 
set Sr being well defined since qr is closed. 

Init: Ef = Xi (K”) , i = 1 , ..., Np, k = 0 Repeat 
For z = 1, ..., Np 

rf+i = (I, (i?-i (xfj n ... n R~i^ J)) 

End For 

k = k + 1 

Until a set {Ei}.^j^ of fixed points has been found. 

If such a set cannot be found, then EXIT. 

End SUB 

SUB Strongly.Connected.Simp{ F ) 

For all qi G Ip, let {qi,,, ..., qi„,i} C Q be the set of nodes that are successors 
of the node qi. In general, the set {qi, , ..., qi„,i} may contain nodes not belonging 
to F. Find the maximal robust controlled invariant set E* with respect to config- 
urations {qi,S (qi)) , ■ ■ • , {qNp,S (qNp)) and constraint (3) contained in the set 
Ei, n Ei^ n • • • n Si^,. if such a set cannot be found, then EXIT. Let Ei = S* , 
z = I, ..., Np and return. 

End SUB 

Proposition 1. Consider a switching system with E described by a general con- 
nected FSM. Let Si, z = I, • • • ,N he the sets found by Algorithm I. Ifq{to) = qi, 
for some z = I, • • • , N, then Xg = Ei. 

The structural approach proposed with Algorithm 1 decomposes the original 
problem into a number of different sub-problems, each consisting of finding a 
maximal (robustly) controlled invariant set in a given constraining set for a con- 
tinuous state dynamical system or for a finite set of dynamical systems. There 
are essentially two levels of computation involved: a higher level corresponding 
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to the steps of Algorithm 1, and a lower level, called by the higher level, cor- 
responding to the computation of the invariant sets. The structure of the FSM 
has been exploited in order to achieve maximal computational efficiency when 
solving the continuous sub-problems. Assuming that the lower level converges 
appropriately, Algorithm 1 is guaranteed to converge to the exact solution, if it 
exists. Moreover, in the case of acyclic FSM or if = 0 for i = l,...,Abr, it 
converges to the exact solution in a finite number of steps. In order to better 
understand the improvements in computational efficiency, we compare our pro- 
cedure with the one described in literature (see e.g. [21]), on a switching system 
described by: 

Qi — *■ Q 2 — *■ — *■ Qn 

Applying our procedure to this example, we have: 

For z ^ A^ — 1 to 1 

End For 

The solution needs N iterations, and the computation of N controlled in- 
variant sets. Applying the procedure described in [21] to the same system, we 
have: 

lF*+i = IF* - Reach{Pred{W^),%) 

where IF* is the safe-set approximation found at iteration i, in the mixed 
discrete-continuous state space, and: 1F° = {{Qi, n- 

Let: wl. = The preceding computation is equivalent to: 

For i ^ 1 to N 

For fc^ltoA^ — z+1 
wi=ik{wl-^ n ... n 

End For 
End For 

In this case, the procedure of [21] needs the computation of N{N— l)/2 max- 
imal controlled invariant sets. The two procedures require the same computation 
only in the case of a switching system described at the top level by a strongly 
connected FSM. 

Algorithm 1 can be extended to the more general class of hybrid systems, 
where “invariance” conditions and controllable transitions are present in addi- 
tion to uncontrollable switchings. We give two examples to illustrate how the 
procedure works. The extension to more complex FSM topologies can be han- 
dled similarly and can be found in [3]. Assume to have a system described by 
the following discrete structure: 

Qi — *■ Q 2 

where the transition from Qi to Q 2 is forced when the system ceases to satisfy 
an “invariance” condition of the form: x € Fi. 
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~ The safe set for configuration Q2 is given by: S2 = 2^2 (R"), i.e. the maxi- 
mal controlled invariant set, with respect to configuration Q2 satisfying the 
output constraints. 

— The safe-set for configuration Qi is given by: 

Si = Xi(A) U (A, n Ri2\S2)) 



where: 

{ X : 3 tp, u{») e U such that: if for some '1 

to < tp x(to) = X, x(t) G An f 2 ix > U n f 2 ix) 
for to <t < tp and x{tp) G B j 

and x{t) is the state evolution for configuration Qi. 

The interpretation for the formula above is as follows: for each x in the safe 
set for Qi, there should be a control law such that the corresponding trajectory 
never goes outside the invariant set Fi, or, if it does, the reset function maps the 
actual state a; in a “safe state” for configuration Q2- Moreover, the constraints 
on the output must always be satisfied. 

The case of controllable transitions follows the same logic. Suppose that the 
transition from Qi to Q2 is completely controllable. Then: 

— the safe-set for configuration Q2 is given by: S2 = F2 (R”); 

— the safe-set for configuration Qi is given by: 

A = h (R”) U Zi (R", {S2)) 

In this case the trajectory can or cannot be entirely in the controlled invariant 
set Xi(R"), but if it goes outside it must end into a set which is the reverse image 
of the safe-set for Q2, and when it happens the discrete controller forces a switch 
from Qi to Q2- Also in this case constraints on the output must constantly be 
satisfied. 

4 Construction of Invariant Sets 
and Convergence Properties 

In general, the computation of a controlled invariant set is an open problem. 
In fact, while conditions such that a given set enjoys the controlled invariance 
property have been extensively studied in the context of viability theory (see 
e.g. [ 1 ]), there are no implementable results applicable to general nonlinear sys- 
tems. In the case of continuous-time linear systems, Dorea and Hennet in [ 13 ] 
characterize controlled invariance for general convex polyhedral sets with com- 
putable conditions. In addition, they show that no iterative formulas exist to 
exactly compute some maximal controlled invariant set for general continuous- 
time systems. Fortunately, methods for the computation of maximal controlled 
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invariant sets for discrete-time linear systems and polyhedral constraining sets 
are well known in the literature (see e.g. [6], [11], [12], [15], [17]). In all of these 
papers, recursive algorithms are given that converge to the exact required set, 
if it exists. Hence linearizing and discretizing general dynamical systems is cer- 
tainly a feasible path towards a computationally efficient approach for maximal 
controlled invariant set computation. 

Given the i-th discrete-time linear system: 

x{t+l) = Ai X (t) + u (t) + Fi6 (t) 

this set can be computed by means of the following backward procedure (see e.g. 

[ 6 ]): 

Algorithm 2: Maximal Controlled Invariant Set 

1° ^ yl 

Repeat 

Until 



where A is the initial constraint set in the state-space, and: 

n]^{A) = {a; e M" : : AiX + BiU + Fi6 G A and y £ 0^5 & U^} 

In general, this recursive algorithm converges to the solution asymptotically. 
If A, 17, and are polyhedral, so are the sets but not necessarily the limit 
of the sequence for k which tends to infinity. In general, the maximal controlled 
invariant algorithm is not guaranteed to terminate in a finite number of steps. 
To the best of our knowledge, the only result that gives a sufficient condition for 
controller synthesis decidability can be found in [20]. 

At each step, the computation of in Algorithm 2 involves a projection pro- 
cedure and the elimination of redundant constraints. The approaches presented 
in the literature essentially differ for the algorithm used to project a polyhedron 
on a given subspace. The classical Fourier Motzkin elimination method and its 
modified versions (see [19] and [17]) can be used to perform this task. In our 
example, we adopted the algorithm developed in [10], which has the advantage 
of identifying and removing redundant inequalities at every step. To the best of 
our knowledge, a systematic comparison among the projection algorithms has 
not been done so far. 

In this section we make the following assumptions: 

~ In each configuration of our switching system, the dynamical system is a 
discrete-time linear system, i.e. we have: 5 : Q — > Sd , where 



S{q^) = 



x{t+l) = Ai X (t) + Bi u (t) + Fi6 (t) 
y {t) = Ci [x {t) + DiU (t)) 



z G (/ 
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— Constraints are given as linear inequalities and affect both the continuous 
state and the control input of the system: 



r WiX < M, (a) 

\ DiU < (b) 



Since the output y represents the controlled output, we can choose the ma- 
trices „ and „ so as to specify both constraints on the state 

and control in terms of constraints on the output: 



Ci = 

Then (5) is equivalent to: 

y G f2 where: f2 = 



7 O' 


and: Di = 


'0 O' 


0 0 




0 I 



y ■ 



W, 

D, 



y < 



M, 

di 



The continuous disturbance i5(.) takes values in a bounded polyhedral set (poly- 
tope), Ud, that is a set described by linear inequalities: 



GS<H 



The reset function R{{qi,a,qj),x) is an affine function: 

R{{qi, a, qj), x) = Rijx + 

In order to apply the algorithm described in Section 3, we need to give a numer- 
ical implementation of the following operators: 

a) b) c) X,(.); 

The computation of the set R~j^{A) is straightforward if is a polyhedral set, 
i.e., A = {x : Vx < N}. In fact we have: 

R~/{A) = {x :V (RijX + Pij) < N} = {x : VRijx < N - Vpij} 

The implementation of both operators and Ti(») is based on the imple- 

mentation of the operator: 

for Z\ = 1. 



Then, 



nl^(A) = {x e M" : : AiX + BiU + Fid S A and y G G U^} 

The procedure used to compute the set Gl^(A) can be found in [3]. At each 
step, redundant inequalities are eliminated by an appropriate algorithm. The 
numerical implementation of is the composition of the operator 

repeated A times. If the set A is polyhedral, the set flA{A) is also polyhedral 




Invariant Sets and Control Synthesis for Switching Systems 



69 



and, given the implementation of it is described by a set of non-redundant 
linear inequalities. 

The computation of the maximal controlled invariant set contained in a poly- 
hedron A is similar, since we repeatedly apply halting when the set found 

at iteration k is the same as the one found at iteration k — 1. 

Since, in general, the maximal controlled invariant algorithm is not guaran- 
teed to terminate in a finite number of steps and, in addition, it is not possible 
to give an a priori bound on the number of inequalities characterizing it, a 
basic question is whether it is possible to find a good approximation. If the al- 
gorithm terminates, then the set found after the last iteration is the maximal 
controlled invariant set. Otherwise, the algorithm, as it progresses, computes 
an increasingly better approximation of this set. Further, since the numerical 
implementation of the operator eliminates redundant inequalities, the al- 

gorithm gives also a representation of the set 2i(») that is minimal in terms of 
the number of linear inequalities. 

Algorithm 2 builds, recursively, outer approximations of the maximal con- 
trolled invariant set. Hence at each step, although we can go as close as we 
want to the exact solution, we obtain sets that are not invariant. Our idea is to 
construct also inner approximations to the maximal controlled invariant set by 
building recursively sets that, at each step, are controlled invariant. Then, we can 
approximate the maximal controlled invariant set with the inner approximation 
and combine both outer and inner approximations to quantify the error associ- 
ated with the inner approximation. The evaluation of this error is of paramount 
importance in our approach, where the error is not confined in the computation 
of just one maximal controlled invariant set, but it propagates backward, as the 
higher level algorithm proceeds, and one could easily find an empty set at some 
step of the recursion, even if the given problem has a solution. 

Suppose that no disturbance is active on the system (Ud = {0}). For re- 
cursively constructing an inner approximation, if the set A is convex, bounded, 
containing the origin in its nonempty interior, the algorithm used is identical to 
Algorithm 2, except for the initialization of which has to be set equal to a 
starting controlled invariant set and not to the set of state-constraints: 

— if the system is controllable, set = {0}; 

~ if the system is asymptotically stabilizable, set = A, where A is a con- 
trolled invariant set with non-empty interior contained in the set of con- 
straints A . 

For finding the controlled invariant set A, an ellipsoidal A-contractive set 
contained in the set of constraints can be obtained easily. Then, by applying the 
procedure described in [6] for a value X' of the parameter A greater than A, a 
polyhedral A-contractive set. A' < A < 1, containing the maximal A'-contractive 
set is obtained in a finite number of steps. This set is obviously controlled invari- 
ant and has nonempty interior, because the maximal A'-contractive set contains 
the ellipsoidal A-contractive set. Alternatively, an invariant controlled set con- 
tained in the constraint set can be obtained by using the constructive result of 
[14]. 
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We can show that [3] the algorithm terminates if a fixed point is found and 
converges to the maximal controlled invariant set. This result, in the case of a 
controllable system, generalizes the one in [16], where the invertibility of Ai is 
required. 

The maximal controller is the set of all control laws u(-) that make all con- 
straints satisfied, for any allowed switching and disturbance, starting from the 
set Xo, found by Algorithm 1. When the discrete location is qi, if we apply 
a control law in the maximal controller, the state, starting from Si, reaches a 
maximal controlled invariant set in Ai steps of time. It remains in this set until 
a switching occurs (since the chosen control makes this set invariant). A con- 
trol law which makes a given set invariant depends on the set itself. If the set 
is a polyhedron containing the origin and if the dynamical system is a linear 
discrete-time system, it can be a piecewise linear state feedback control law that 
can be determined by using a technique introduced in [15] for the poly topic case 
and generalized to the polyhedral case in [13]. The polyhedron is partitioned in 
a certain number of subsets, and for each subset a different linear state feedback 
law is applied. The number of these subsets may be arbitrarily large, and hence 
there is no bound on the complexity of the control law, which has to be com- 
puted on-line. To reduce the on-line computational effort, the result of [8] can 
be used. Blanchini shows that if a controlled invariant polytope is approximated 
by a suitable smooth domain, a simpler control law exists. 

If the dynamical systems corresponding to each location of the FSM are con- 
tinuous and not discrete-time, we have to choose the discrete-time system that 
corresponds to the given continuous-time one so that a precise relation can be 
established between the invariant sets computed as previously illustrated and 
those of the continuous-time system. One way of obtaining such a relation is to 
consider the Euler Approximating System (EAS) (see [8] and [7]). As the param- 
eter T characterizing the Euler Approximation tends to zero, better and better 
approximations are obtained for the maximal controlled invariant set of the 
continuous-time system. Moreover, these sets are invariant for the continuous- 
time system. The same result holds if the sampled-data system is used, but in 
that case the approximations are not invariant [8]. 

5 Application to Engine Idle-Speed Regulation 

The idle-speed control problem deals with the task of maintaining, while in the 
idle mode, the engine speed into a given range, rejecting torque disturbances due 
to accessory loads (such as the air-conditioning system and the steering wheel 
servo-mechanism), preventing the engine from switching off. The power-train 
model used for idle-speed control is: 

ji> = ttmnp + braa 

\ h = + 6„ (T - Tload) 

where n is the engine speed expressed in RPM (Revolutions Per Minute); p is 
the manifold pressure expressed in mbar; J^q is the momentum of inertia for the 
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transmission chain (kg m^); a„i, bm, cLn, bn, are constants; T = kiij (AV)p is the 
torque produced by the engine given the spark advance angle AV, the efficiency 
function rj (AV) and the constant ki; Tlqad is the torque disturbance from 
accessory loads. The two control inputs are the throttle opening angle a and the 
spark advance angle AV. Clutch insertion or release has the effect of modifying 
the parameter J^q, the momentum of inertia, causing a sudden unpredictable 
change in the power-train parameters. 

We assume that no information is available about minimum permanence 
times in each configuration, so we let Ai = 0, i = 1,2. Moreover, the reset 
function is the identity. 

The problem is to find under which conditions it is possible to maintain the 
engine speed into the desired range 800 ± 30 RPM, satisfying the constraints on 
control inputs: 0° < a < 20°, 0° < AV < 20°. 

Defining the output of the system as y = [n p a AV ] , we can express the 
constraints on the state and the inputs as output constraints. By applying the 
method described in Section 4, we determined the safe set and we found that 
the maximum value allowed for the continuous disturbance T^oad is 12 Nm. 
The maximal controller is described by linear inequalities, where the bound 
vector depends on the state. A controller may be chosen among all possible 
ones by introducing an optimality criterion. Simulations carried on the switching 
nonlinear model show the effectiveness of the proposed approach (see [3] for more 
details on this application). 

6 Conclusions 

We proposed a structural procedure for the determination of the maximal safe 
set for hybrid systems. While demonstrably more efficient than the elegant proce- 
dure of [21], the procedure still suffers from computational complexity stemming 
from the computation of maximal controlled invariant sets of general dynamical 
systems. The procedure is made computationally more appealing by linearizing 
the nonlinear dynamics and using a discrete-time equivalent model, since proce- 
dures for the computation of maximal controlled invariant sets for discrete-time 
linear systems are well-known in the literature. Even for this case, the procedure 
for the determination of the maximal controlled invariant set may not converge 
in a finite number of steps. We propose an inner approximation algorithm that 
together with the classical outer approximation yields tight bounds for an error 
due to the truncation of the procedure after a finite number of steps. The theory 
has been applied to idle-speed regulation in engine control to demonstrate its 
power. 
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Abstract. A general verihcation algorithm is described. It is then shown 
how ellipsoidal methods developed by A. B. Knrzhanski and P. Varaiya 
can be adapted to the algorithm. New numerical algorithms that com- 
pnte approximations of unions of ellipsoids and intersections of ellip- 
soids and polyhedra were developed. The presented techniques were im- 
plemented in the verification tool called VeriSHIFT and some practical 
results are discussed. 

Keywords: hybrid systems, verification, reachability analysis, ellipsoidal 
approximations. 



1 Introduction 

A number of application domains, such as car manufacturing, robotics, chem- 
ical process control, or avionics, involve controllers, consisting of: (a) a set of 
sensors and actuators, representing the interface between the controller and its 
environment; (b) a control logic (implemented as one or more circuits or as one 
or more pieces of software running concurrently), which represents the way the 
controller should act on the environment. 

A promising model for describing such systems is hybrid automata [8] . Hybrid 
automata are finite-state machines equipped with continuous variables. Each 
discrete state of an automaton has a system of differential equations that govern 
its continuous variables. Most correctness criteria for such systems can be stated 
as a safety property: the system must never reach an “unsafe” (or a “bad”) state. 

Ensuring correctness of the model is often not a trivial task. Simulation of the 
system is not adequate, since it can only help examine a limited number of tra- 
jectories. Analytical methods are often not applicable, considering the complex 
interaction of continuous and discrete dynamics. An alternative is reachability 
analysis. It consists of computing the set of all reachable states of the system and 
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then checking that no “bad” state belongs to the reachable set. The reachability 
problem has been shown to be undecidable, even for models of hybrid automata 
with simple dynamics (e.g., x G [a, 6]). Moreover, the so-called state explosion 
problem (the machine representation of the set of reachable states is too large) 
often limits the applicability of the method, even for decidable sub-classes of the 
model. 

Approximations have been used as a remedy to both the undecidability and 
the state-explosion problems. Computing an over- or under-approximation (i.e., 
external or internal approximation) of the exact set of reachable states can be, 
first, decidable, and second, less expensive, in terms of time and memory. The 
price to pay is accuracy: what does it mean for a “bad” state to be reachable in 
the approximative analysis? 

This paper presents a new reachability technique for systems of hybrid au- 
tomata with linear dynamics, expressed as differential inclusions: x G Ax + U. 
The basic model of hybrid automaton and its semantics are presented in sec- 
tion 2. 

The algorithm performs reachability analysis for bounded time. Reachabil- 
ity for bounded time means that the set of states reachable in A time units is 
computed, where Z\ is a parameter supplied by the user. The skeleton of the algo- 
rithm, correctness, and trade-offs between accuracy and efficiency are discussed 
in section 3. A generalization of the algorithm is presented in [1]. 

The algorithm is based on the ability to approximate: (a) the reachable set 
of a linear differential inclusion (time propagation); (b) intersections of convex 
sets; (c) unions of convex sets; (d) linear transformations and geometric sums of 
convex sets. 

Among methods of reachability analysis are those based on ellipsoidal tech- 
niques. The presented work is an attempt to use some of the methods described 
in [5,6]. 

New methods for computing over-approximations of unions of ellipsoids and 
intersections of ellipsoids and polyhedrons have also been devised. They are 
presented in section 4. 

These reachability techniques have been implemented in a prototype tool 
called VeriSHIFT (section 5). The tool accepts systems of hybrid automata, 
communicating by input/output variables and synchronous message passing. 
Dynamic creation and reconfiguration of automata is also supported. 



2 The Model 

In this section we present the model of a single hybrid automaton. We consider 
the extension to systems of communicating hybrid automata in [1]. 

Preliminaries Let R be the set of real numbers, the set of m x n real 

matrices, C” the set of convex closed subsets of R”, and Ch” the set of convex 
compact subsets of R". B”(a;), the hall of dimension n with center x and radius 
e, is defined to be the convex set {y G R” | \x — y\ < e}. 
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Given a set P G C"' and a matrix A G AP is the linear transformation 

of P, that is, a set from C™ defined as AP = {Ax \ x G P}. 

Given two sets Pi, P2 G C”, let Pi + P2 denote the geometric (Minkowski) 
sum of Pi,P2, defined as: 



Pi + P 2 = {x \ 3xi G Pl,X2 G P2,X = Xi + X2}- 

A flow in R” is defined as a triple F : {A,I,U), where A G R”’" and / G 
C, U G Cb". F defines the following system of differential equations: 

x(t) = Ax(f) + u(t) 
x{t) G I, u{t) G U. 

Given points xq,xi G R", we say that xi is F -reachable from xq at time t, 
denoted xq-^f xi, if there exist functions of time x{-) and u{-) such that a;(0) = 
xo, x{t) = xi, and for all r € [0, t], x{t) G I, u{t) G U and x{t) = Ax{t) 

Given a set Xq G Ch", the reachable set of flow F from Xq at time t, denoted 
Xf{Xq, t), is the set of all points that are F-reachable from points of Xq at time 
t. 

A Hybrid Automaton. We define a hybrid automaton A with linear differential 
inclusions to be a tuple {Q, X, F, T, G, R), where: 

— Q is a finite set of discrete states (or locations, or modes). 

— X is a set of n continuous variables taking values in R. 

— F : Q ^ R”^” xC” xCfe” associates with each discrete state q a flow {A, I, U). 
I is called the invariant of q and will be denoted as I{q). 

— TCQxQisa set of discrete transitions . 

— G : T ^ associates with each discrete transition a guard. 

— R-.T ^ pnxn ^ associates with each transition a pair {B, P). This pair 
defines the reset of the continuous variables^: x := Bx + P. 

Given a discrete state q, let out(( 7 ) be its set of out-going transitions, {{q,q') G 
T}. 

We now turn to the semantics of a hybrid automaton like A. A state of A is 
a pair (q,x) G Q x such that x G I{q). 

Given a state {q, x) and a delay (5 G R, we say that there is a time transition 
from (g, x) to a state {q, y), denoted (g, x) -^{q, y), if x-^F(q) U- 

Given a state {q,x) and a discrete transition a = (g, g') G T, such that 
R{a) = {B, P), we say that there is a discrete jump from (g, x) to a state (g', y), 
denoted (g, x) (q', y), ii x G G{a), y G I{q') and y G Bx + P. 

^ If P contains a single point, P = {y} the reset is deterministic, that is, each x is 
mapped to a unique x' = Bx + j/. If P is not a singleton, then the reset is non- 
deterministic. 
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Reachability. Given a set of initial states Sq, we say that a state s is reachable 
from So if there exists sq G Sq and a sequence 



So Sj 



/ <^1 



^2 / a2 

Si ^ S2 — ^ 



Sk 



( 1 ) 



such that s'j^ = s. The sequence (1) and the corresponding trajectory of continu- 
ous variables x{-) is called an execution of the hybrid automaton, and k is called 
the length of the execution. We say that s is reachable from So in time A if 



i5i -l- (^2 + ■ ' ' + ^ A. 



Given a discrete state q, we say that q is reachable from So (in time A) if 
there exists a state (q,x) which is reachable from Sq (in time A). 

3 Reachability Using Convex Approximations 

Given an automaton A and a set Sq of initial states, we want to verify whether 
a discrete state qbad is not reachable from So in time A. 

In this section we describe the skeleton of the reachability algorithm. The 
algorithm is based on the ability to: (a) effectively represent convex compact 
sets X G Cb^', (b) compute an over-approximation Xp{Xo^ t) A Xp{Xo, t) of 
the reachable set of a linear flow F from a convex compact set Xo at time t; 
(c) check whether the intersection of two convex sets is non-empty; (d) compute 
over-approximations of intersections, unions and geometric sums^ of convex sets; 
(e) compute linear transformations of convex sets. Section 4 deals with points 
(a) - (d) in detail. In this section, we assume that an effective representation of 
convex sets and the above operations are available. First we present the basic 
structure of the reachability algorithm. Then we discuss alternatives and their 
impact on accuracy and efficiency. 



3.1 The Basic Algorithm 

The algorithm maintains a table T of tuples of the form: {q, X, t), where q G Q, 
X G Cb" and r G [0, A], (q, X, r) is supposed to represent a set of unexplored 
states {q,x),x G X. A state s = (g, x) is unexplored in the sense that qbad might 
be reachable from s in time A — t. An invariant of the algorithm is that if a 
state (g, y) is reachable in time A then at some point the table will contain a 
tuple (g, X, r), where t < A and, either y G X, or there exist x G X and t G R 

such that x ~^F(q) U- 

T is initialized to So (the set of initial states), such that for all {q,X,r) G 
T, T = 0. The algorithm essentially repeats three steps. First, it chooses an 
unexplored tuple {q,X,T). Second, it propagates X in time, until time reaches 

^ The geometric sum P\ + P2 can be computed exactly if at least one of Pi , P2 is a 
singleton. Consequently, the reset of a set X, BX + P, can be computed exactly if 
the reset is deterministic (i.e., P is a singleton). 
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A; meanwhile, it computes the intersection of the reachable tube from X with 
each of the out-going guards of q. Third, for each intersection V with a guard, 
the algorithm computes the reset of V with respect to the corresponding discrete 
transition, and adds a new (unexplored) tuple to the table. 

The second step involves computing the reachable set of F{q) from X at 
time t, for t G [0, Z\ — r]. Since it is not possible to compute this set for infinitely 
many time values, we have to discretize time, that is, we compute Xp{Xo, t) for 
t = kS, where fc = 0, ..., • The time step i5 is a parameter of the algorithm, 

given by the user. In order not to “miss” a guard during the propagation of X 
in discrete time steps, we “enlarge” the reachable set at each time step by a ball 
of radius e (see step 2, below), e can be effectively computed as a function of 
F(q) and S, so that correctness of the over-approximation is ensured: 

Lemma 1. The following estimate is true for all t G [0, i5].' 

Xf{Xo, t)CXo + B,{0) (2) 

where Xp{Xq, t) denotes the reachable set of differential inclusion 

F:xGAx + U, U G Cb^, 

Se(0) G C{,” is a ball of radius e with the center in 0, and 

g = (e^Ai _ e^^^NuS, 

Na = Pll = max ||Ha;||, 

IHII=i 

D = max ||a;||, Nu = max ||m||. 

X^Xq U^U 

Proof To ensure that inclusion (2) holds it is enough to take e equal to the 
Hausdorff semidistance between Xg and Xf{Xq, t): 

e = h+^Xp^Xo, t), Xq) = max min \\x — y\\. 

y&Xp{Xo,t) xeXo 

Then it is not difficult to see that hp{Xp{Xo, t), Xq + f?e(0)) = 0 which implies 
Xp{Xq, t) CXo + B,(0). 

Let us estimate this Hausdorff distance: 

h+{Xp{XQ, t), Xq) = max min ||?/ — a;|| 

yG.Xp{XQ , t) x^Xq 

= max max min \\xp(y, t, u(-)) — x\\ 

y^Xo u{-)gU x^Xq 

< max max \\xp{y, t, u{-)) — y\\ 

y^Xo 

= max max \\e^*y — y+ / e"^^*“®^u(s)ds|| 

y^XQ u{-)^U Jq 

< max ||(e"^* — /)y|| -I- max || [ e^^‘“^^M(s)<is|| 

yeXo u(-)GU Jq 
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< max ||(e"^* — /)y|| + [ max||e^^‘ 

vexo"^ ' " Jo ueu " 

< - 1)D + e^^^NuA. 



Now we are ready to detail the steps of the algorithm: 

1. If the table T is empty, stop and announce that qtad is unreachable in time 
A. Otherwise, if there exists a tuple {qtad, -) G T, stop and announce that 
qbad is possibly reachable in time A. Otherwise, choose a tuple (g, X, t) G T 
with minimal r, remove it from the table and proceed to step 2. 

2. Let (g, X, r) be the tuple chosen in step 1, F = F{q) and out(g) = {m, ..., ai}. 

Also, for i = let Oj = (q,qi), R{ai) = and Gi = G{ai). 

(a) Compute Xp (Aq, kS) + B”(0), for k = 0 , ..., m, where m is the minimum 

between and the smallest k < such that (A^(Aq, k6) + 

B^(0)) n/(g) = 0. 

(b) For each i = 1 , ..., I, let kt be the first time the reachable set intersects the 
guard Gi, that is, ki = min{/c | (0 < fc < m) A {{Xp{Xo, k5) + B”(0)) n 
Gi yf 0)}. If the reachable set never intersects Gi, we set ki = m + 1. 
Let Ti = kiS. If Si yf 0 (i.e., the reset is not constant), then we compute: 

m 

14 ^ U {{X^{Xo, k6) + B^(0)) n G,) . (3) 

j=ki 

That is, Vi is (an over-approximation of) the union of intersections of the 
reachable set and the guard at times t > Ti. If Bi = 0 the computation 
of Vi is unnecessary. 

3. For each n < A, computed at step 2, we add a tuple (g^. A', r -I- Ti) to the 
table T, where, 

X' = R, if B, = 0, 

A' A BiVi + Pi, otherwise. ' ' 

Go back to step 1. 

We should point out that in step 2(b), we do not need to compute (an over- 
approximation of) the intersection of the reachable set and the guard at each 
time step to check whether it is non-empty. Instead, we can have a procedure 
that checks, given two convex sets, whether their intersection is non-empty. If it 
is, then we can compute it. 

Correctness and Termination We now state the main properties of the algorithm. 

Lemma 2. Let Aq G C”, G G C”, F : x{t) G Ax + U - some linear flow and 
S > 0. If e is choosen accordingly to lemma (1), then 

A k 

U (A+(Ao, r) n G) c U((A+(Ao, i5) + BflQ)) C G), 

r=0 i—0 



( 5 ) 
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where 

k&Z,{k-l)5 < A< k 6 . 

The proof is a consequence of lemma (1). 

Theorem 1 . If the algorithm terminates doing reachability analysis of a hybrid 
automaton A for time horizon A and reports that the state qbad is unreachable 
(step 1), and a state {q* , x*) is reachable at time t < A as result of a sequence 
of time and discrete transitions ending with a discrete transition, then at some 
step of execution the table T contained a tuple {q*, X, r) such that x* € X and 

T <t. 

Proof Let us suppose that 

Si / ai S 2 / a.N-1 Spi ! apf 

So Si >Si 7 ^ > SAT_i ^ > SAT, ( 6 ) 

F(qo) F(gi) F{qN-i) 

SN-i = {q'j x'), s'^ = {q', x”), sn = {q* , x*), 61 + 62 ~\ \~ Sn = t, 

is an execution of length N that leads to the state {q* , x*) at time t. Let Gi 
and Ri = {Bi, Pi) denote the guard set and the reset relation that correspond 
to transition q' — > q* in sequence ( 6 ). 

The theorem is obviously true for the states that can be reached by executions 
of length 0 . 

Suppose that the theorem is true for states that can be reached in time A by 
executions of length N—1. Let us prove that it is true for executions of length N 
as well. If the theorem is true for executions of length — 1, then at some step 
of execution the table T must contain a tuple r' = {q' , X' , t') such that x' G X' 

and r' < (5i + 152 H \~Sn-i = t' . Since the tuple r' appeared in the table, step 

2 of the algorithm was applied to it. Using lemma (2) we can conclude that the 
set Vi constucted by (3) contains the point x” . Hence, the set X( resulting from 
the reset relation (4) contains the point x*. That proves the theorem. ■ 



Theorem 2 . If the algorithm terminates and reports that the state qbad is un- 
reachable in time A (step 1), then the state qbad is unreachable in time A. 

The proof is a direct consequence of theorem (1). 

Termination of the algorithm is not guaranteed for systems that may present 
so-called zeno behavior: an infinite number of discrete jumps in a finite amount of 
time. The following theorem states that termination is guaranteed when the time 
“consumed” by each loop of discrete transitions of the automaton is bounded 
from below by a positive number (in our case, at least by i5). 

Theorem 3 . If for any loop ai = (91,92), 02 = (92, 93 ), •••, Ofc = (9fe,9i) G T, 
there exists i G [ 1 ,A:] such that the following conditions are satisfied: 
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1. R{ai) = (0, P), that is, the reset of ai is constant. 

2. {P + B”(0)) n G{ai+i) = 0 (by convention, ak+i is taken to he a\). 

then the algorithm terminates. 

The proof is quite obvious and is based in the fact that any cycle in the transition 
graph takes at least one intergration step 6. 

3.2 Possible Modifications 

Alternative choices could be made at some points in the algorithm. We discuss 
these possibilities below and comment on their impact on the accuracy and the 
efficiency of the algorithm. 

1. If the table contains two tuples {q, Xi, n) and {q, X 2 , T 2 ) with the same 
discrete state, then we replace these tuples by a single tuple {q, Xi U A 2 , 
min{ri, T 2 }). This decreases the size of the table and results in fewer tuples 
to be explored. On the other hand, since we can only compute an over- 
approximation of the union Xi U X 2 , the accuracy of the algorithm might 
be compromised. Correctness is not affected. 

2. At step 1, instead of removing the chosen tuple {q, X, r) from the table 
T we mark the tuple explored. Only unexplored tuples are chosen in step 
1. Moreover, before adding to T a new (unexplored) tuple {q, X', t'), T is 
searched for a tuple {q, X", r") such that X' C X” and r' > t". If such a 
tuple exists then {q, X' , r') is not added. The status of {q, X" , t") (explored 
or not) is not changed. The correctness of the algorithm is not affected. The 
size of T could increase since explored tuples are not removed. On the other 
hand, new tuples are not added to the table when not necessary, which results 
in fewer tuples to explore and shorter running time. 

4 Ellipsoidal Approximations 

The reachability algorithm described in the previous section can work with any 
representation of convex compact sets, as long as the operations used by the 
algorithm can be performed effectively on the chosen representation. 

The verification tool described here uses ellipsoidal techniques for approxi- 
mation and reachability analysis. One of the advantages of ellipsoidal methods 
is that an ellipsoid in Ch” can be described as a pair {x, P) G R" x that is, 

using only O(n^) space. Time complexity of ellipsoidal operations is also poly- 
nomial.^ The numerical methods that have been used are directly taken from or 
based on the results described in publications [5,6]. 

In these works it is shown that ellipsoidal over-approximations of reachable 
sets can be expressed through ordinary differential equations with coefficients 
given in explicit analytical form. Other results include parametric representa- 
tion of ellipsoidal over-approximations of geometric sums and intersections of 

® As a comparison, the worst-case complexity of polyhedral operations is exponential. 
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ellipsoids. The reader is referred to the above-mentioned publications for the 
details of the methods. 

Here, we present new techniques that we have developed for operations on 
ellipsoids and polyhedra and for unions of ellipsoids. 

Definition of Ellipsoids Let {I, x), l,x G R", denote the inner product of I and 
X. An ellipsoid £{p,P), p G R”, P G R">^", P = P' > 0 ^ is a convex compact 
set described by the support function^ 

p{i\£{p, p)) = {I, p) + {I, piy/^. 

If the matrix P is non-degenerate, the ellipsoid £{p, P) can alternatively be 
defined as a level set of a quadratic function: 

P) = {x\ {x-p, P~\x-p)) < 1}. 

Approximation of Unions of Ellipsoids The reachability algorithm of section 3 
uses union of convex sets in step 2(b), equation (3). Union is needed also if 
tuples {q, Xi, n), {q, X 2 , T 2 ) G T are replaced by a single tuple {q, Xi U A 2 , 
min{Ti, T2}), as described in one of the alternative heuristics. Here we describe 
the algorithm for over-approximating the union of two ellipsoids by an ellipsoid. 
Such an algorithm should be efficient, since it is likely to be the bottle neck of 
the basic verification algorithm. It should also exploit the fact that the reachable 
set changes only slightly in one time-propagation step. 

Let us suppose that we want to approximate £{p, P) Uf (r, R), where P and 
R are non-degenerate matrices. 

The algorithm builds an increasing sequence of ellipsoids: 

P) = £{Po, Po), £{pi, Pi),--,£{Pk, Pk), (7) 

until £{pk, Pk) D £(r, R). 

£{pi+i, Pi+i) is obtained from £{pi,Pi) as follows. Given Pi and R, we com- 
pute matrices Li, Vi, Di and Si. Li is a lower triangular matrix that is the 
result of Cholesky decomposition of the matrix P~^: PiL) = Pff^. U is a 
matrix of eigenvectors and Di is a diagonal matrix of eigenvalues of matrix 
Ci = L~^ R~^ L'~^ such that Ci = ViDiVf. We denote 

Si = L,V (8) 

and y* = S[{r -pi). 

Then we find a vector x* that is the solution to the non-convex optimization 
problem 

Ji{x) = {x - Ui, Di{x - Di)) max 

^ P' is the transpose of matrix P. Similarly, x' is the transpose of vector x. 

® The support function p{l\ X) of A € Cb" is defined as p{l\ X) = max{l, x). Inversely, 

x^X 

a support function uniquely defines a convex compact set. 
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with the constraint ||a:|| < 1. In [12] it is shown how the problem can be solved 
and it is proved that the result is the global maximum. 

We will denote I* = ■ 

II II 

If Ji{x*) > 1, it means that £{pi, Pi) 3 £(r, R) and the algorithm terminates. 
If Ji{x*) < 1, we compute: 

Pi+i = 5'"^ {vi + +Pi, (9) 



and 

P.+i = S'-^ (^(1 + a-^)D-^ + (1 + ai)’^xlx*'^ S~\ 

where 

di = l- \j{x*, D~^x*} - {x* , y,) + e, 

OCi 7y 5 

d^ {X*, X*)^ 

and e is any non-negative number. 

Lemma 3. (See [5]) Let £\ = £{qi,Q\) and £2 = £{q 2 ,Q 2 }- 



(10) 

( 11 ) 

(12) 



1. The ellipsoid £ = £{qi + q 2 , Q{!3)), where /? > 0 and 

g(/3) = (i + r')Qi + (i + /3)Q2, 



is properly defined and is an external approximation of the geometrical sum 
£1 + £ 2 , i-e. 

£1 + £2 £{qi + 92 , QiP)) 

for any P > 0. 

2. With vector I G R", |K|| = 1, given, the equality 



p = 



{Qii, l) 
{Q2I, l) 



defines a scalar parameter p, such that 



p{l\ £{qx + q 2 , Q{P))) = p{l\ £{qi,Qi) +£{q 2 ,Q 2 ))- 
(The approximation £{qi + q 2 , Q{P)) touches the exact sum in direction 1.) 



Lemma 4. Given any e > 0, for the ellipsoid £{pi+i, Pi+i) in sequence (1) the 
following holds: 



1 . 



£{jpij Pi) tZ £(^Pi-\-i, Pi^i) , 



(13) 
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2 . 

p{Vl\^{Vi+\-, ^*+i)) = P(l*\ £{r, R)) + e. (14) 

Proof The linear transformation 



x' = S[{x-pi), 

transforms the ellipsoid S{pi, Pi) into the unit ball £(0, /„) = k{x\ {x, x) < 1} 
and transforms the ellipsoid £{r, R) into the ellipsoid £{yi, Di). The next 
ellipsoid in the sequence Pi+i) as specified by (9) and (10) is the result 

of the reverse transformation applied to the ellipsoid 

£' = £ (1 + Oj ^)Di ^ + (1 + ai)-^x*x* ^ 

which, according to lemma (3), is an external approximation of the sum 
£{Vi, Di)+£ (^^x*i, ^x*x*'^ 



It is not difficult to see that 0 G £ which ensures that £' D 

£(yi, Di), and x* G £ ■ Also, the choice of parameters ai and di 

ensures that 



p{x*\£{Q, In)) + e = p 




£{Ui, Di) + £ 





Pix*\£'). 



The later implies (14). ■ 

Theorem 4. For any e > 0 the algorithm always terminates in a finite number 
of steps. 

Proof Because the support functions p{l\£{pi+i, Pi+i)) and p{l\£{r, R)) are 
continuous, for any e > 0 there is (5 > 0 such that for any I : ||^ — ?*|| < <5 the 
following inequality holds 



p{l\£{p^+x, Pi+i)) > p{l\£{r, R)), 

which means that no Ij can belong to the set {l| ||^ — 1*\\ < <5}. If at some step Ij 
belongs to this set, Jj{x*) > 1 and the algorithm terminates. 

There can be only a finite number of U : = 1 such that for any i, j: 

\\li — IjW > 6. Therefore, the algorithm always terminates in a finite number of 
steps. ■ 

In practice, if the ellipsoid £{p, P) has to be extended just “slightly” in order 
to contain £{r, R), which is the case when approximating the union of reachable 
sets at successive time steps, it is likely that the union algorithm terminates 
after a single step, i.e., £(pi, Pi) D £{r, R). 



Here /„ £ r"X" denotes the identity matrix 
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Intersections of Ellipsoids and Polyhedra Guards of discrete transitions are usu- 
ally given in terms of conjunctions of linear inequalities, which define a polyhe- 
dron. Here we discuss a method for approximating intersections of ellipsoids and 
polyhedra. 

In order to approximate the intersection of an ellipsoid £ and a polyhedron 
H, we first compute ellipsoidal over-approximations of the intersection of £ with 
each of the facets of H. Then, we compute the intersection of the resulting 
ellipsoids. Since each facet of H is a half-space, we now show how to approximate 
the intersection of an ellipsoid and a half-space. 

Theorem 5. Suppose that the ellipsoid 

£{<1, Q) = {a;| {x - q, Q~\x - q)) < 1}, 

where Q = > 0, and the half-space 



S = {a;| (6, x) > a} 
have a non-empty intersection. 

1 . Then for any p G [0, ° the ellipsoid 

Q+{p)) = {a;| {x - q+{p), Qf^{p)ix - q+{p))) < 1} 
is an external approximation of the intersection £{q, Q) n S, where 



and 



q+{p) = q + pP ei, Q+Gp) = PCPG 



P = VD^/^B, ei = (1, 0,...,0)^ 



C = 



/A 0 ••• 0\ 

0 /3 ••• 0 



\0 0 ■■■ pj 



^ 1 ^ a' -\-l-2p , a - (6, q) 

Pi / -\\0 t P / . . ^ \ / 



{p-\Y 



D = 



/AiO •••0 \ 
0 Aa • • • 0 



(a' + l)(p-l)2’ 

, V = {vi V2 ... v„), 



(15) 



yo 0 •••A„y 

where Ai,...,A„ are eigenvalues of matrix Q~^ and v\, ...,Vn are correspond- 
ing eigenvectors that are linearly independent and ||ui|| = 1, i=l,n, 

B = (6' 62 63 ... 6„), b ' = 

V{b, b) 

{62, 63,...,6„} is an orthonormal basis of subspace {a;| {x, b') = 0}. 
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2. The ellipsoid S{q+{p), Q+{p)) touches S{q, Q) at point 

X* = B'^D^/‘^V'^{ei-q). (16) 

3. 

f| E{q+{p),Q+{p)) = E{q,Q)nS. (17) 

pe[o, 

Proof The fact that matrix Q (and Q~^ as well) is self-adjoint and positive 
definite gives us the following equalities: 

V^V = VV^ = I, B’^B = BB^ = I. 

Also note that 

= VDV'^. 

Let us apply a linear transformation: 

x = VD~^^'^Bx' + q, (18) 



then the following holds: 

{x - q, Q-'^{x - q)) = 

{VD-^/^Bx', Q-^VD-^/'^Bx') = 

{x', B^D-^/^V'^Q-^VD-^/^Bx') = {x\ x'). 

Thus, transformation (18) converts the ellipsoid S{q, Q) to the unit ball 

Bo = {x\ (x, x) = 1}. 

At the same time transformation (18) converts the half-space S to the half- 
space 

S' = {x'\{x', B'^ D~^^'^V^b) < a—{b, g)} = {a;'| {x' , < a'}. 

Due to the selection of the matrix B-. 

^ ^ B'^p-^^'^V'^b = ei, 

thus 

S' = {x'\ {x', ei) > a'}. 

Now we take an ellipsoid that is defined by the matrix 

//3i 0 ••• 0\ 

0 /3 ••• 0 

Vo 0 ■■■ 13 ) 



C-\p) 
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where j3\ and (3 are determined by (15) and center 

c{p) = {p, 0, 0,...,0). 

If p G [0, the ellipsoid is defined and it is not difficult to see that 

S{c{p), C{p))DBonS'. 

Moreover, 

s{c{p), C{p))ns' = Bons' 

and the ellipsoid £(c(p), C{p)) touches Bq at point ei = (1, 0, 0, 0). 
It is not difficult to see also that 



S{c{p), C{p)) C {(a;i, 0, 0,...,0)|a;i >2p- 1}. 



If p ^ 2p — 1 ^ a'. Also notice that if p = 0, S{c{p), C{p)) = Bq. 

Therefore 

f| £(c(p), C(p)) = Bo n B'. 
pe[o, 2^) 

If we apply the reverse transformation 



x' = B^D^/‘^V'^{x-q), 



to the ellipsoid £(c(p), C'(p)), we will get the ellipsoid £{q+{p), Q+{p)) as de- 
fined above. Properties (16) and (17) will be satisfied and point Ci at which 
£(c(p), C(p)) touches Bq will be converted to point x* as defined by (16) and 
ellipsoids will touch each other at the point x*. M 



Intersection Check The problem of checking whether two non-degenerate el- 
lipsoids E{pi, Pi) and E{p 2 , P 2 ) intersect is equivalent to a convex quadratic 
optimization problem: 

J{x) = (x-pi, Pf^(x-pi)) ^ min 



with constraint 

{x-p2, P2^{x-p2)) < 1 . 

If x* is the solution to the problem and J{x*) > 1, then the ellipsoids do not 
intersect. Otherwise, they do intersect. 

In order to check whether an ellipsoid and a polyhedron intersect, it is possible 
to check whether the ellipsoid intersects with all half-spaces that form the faces 
of the polyhedron. If it does not intersect at least with one of them, the ellipsoid 
does not intersect the polyhedron. Of course, this method is quite coarse but it 
is simple and effective. If the intersection is actually empty and the above check 
did not find that out, that still can be discovered during computation of the 
intersection. 
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5 Implementaion of the Tool 

The techniques are implemented in the verification tool called VeriSHIFT 
The tool is a C++ library that consists of all necessary numerical algorithms: 
ellipsoidal and polyhedral ® representation of convex sets and operations on 
them, reachability algorithms, verification algorithms described in this paper, 
etc. The user of the tool writes C++ code in order to describe a model: for 
each class of hybrid automaton the user writes a definition of a C++ class 
derived from the special class HyhridOhject provided by the library. The model 
can be defined in terms of high level notions such as discrete states, transitions, 
input/output continuous variables, events, bound convex sets, as described in [1]. 
Each of above notions is defined as a class in the library. Actions taken upon 
discrete transitions can be described as C++ functions. 

The library provides the notion of discrete configuration also implemented 
as a class. A discrete configuration contains a set of objects with their discrete 
states, dataflow and configuration connections and is accompanied by a bound 
convex set containing possible valuations of the continuous variables. 

Together with discrete states, continuous variables and events classes describ- 
ing hybrid automata can contain variables of any possible C-| — h type including 
other hybrid automata classes. There is only one requirement: classes have to 
able to create their copies in other discrete configurations. Objects within the 
same discrete configuraion can use any mechanism provided by C-l — h for commu- 
nicating with each other as long as that does not interfere with the mechanism 
provided by the VeriSHIFT library. In a function called upon a discrete tran- 
sition of an object the object can modify its private data, can create/destroy 
other objects or can call methods of other objects. 

Execution of a typical program using VeriSHIFT starts with creating an 
initial discrete configuration: creating an empty configuration, creating new ob- 
jects within the configuration, setting up connections between objects. Once the 
initial configuration is set up, verification is started by calling a special library 
function. 

In order to verify the properties of the model, the user can observe what 
discrete states the objects enter or can assign special actions to transitions they 
are interested in. Also it is possible to examine reachable sets of continuous 
variables at any phase of the execution. 
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Abstract. We consider the synthesis of optimal controls for continuous 
feedback systems by recasting the problem to a hybrid optimal con- 
trol problem: to synthesize optimal enabling conditions for switching be- 
tween locations in which the control is constant. An algorithmic solution 
is obtained by translating the hybrid automaton to a finite automaton 
using a bisimulation and formulating a dynamic programming problem 
with extra conditions to ensure non-Zenoness of trajectories. We show 
that the discrete value function converges to the viscosity solution of the 
Hamilton-Jacobi-Bellman equation as a discretization parameter tends 
to zero. 



1 Introduction 

The goal of this paper is the development of a computationally appealing tech- 
nique for synthesizing optimal controls for continuous feedback systems x = 
f{x,u), by reducing substantially the complexity of the problem. This goal is 
achieved by virtue of recasting the problem to a hybrid optimal control problem. 
The hybrid problem is obtained by approximating the control set U C by a 
finite set S C U and defining vector fields for the locations of the hybrid system 
of the form f{x, a), a G E; that is, the control is constant in each location. The 
hybrid control problem is, then, to synthesize an optimal switching rule between 
locations, or equivalently, optimal enabling conditions, such that a target set 
f2f C 17 is reached while a hybrid cost function is minimized, for each initial 
condition in a specified set 17 C M”. 

Casting the problem into the domain of hybrid control is not appealing per 
se, on the contrary! Algorithmic approaches for solving the controller synthesis 
problem for specific classes of hybrid systems have appeared [8,12] but no gen- 
eral, efficient algorithm is yet available. Hence, to be able to solve the (nonlinear) 
hybrid optimal control problem, we must exploit some additional property. We 
have a feasible and quite appealing approach if we can translate the problem 
to an equivalent discrete problem, which abstracts completely the continuous 
behavior. This translation is possible if we can construct a finite bisimulation 
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defined on the hybrid state set. The bisimulation can be constructed using the 
geometric approach reported in [4] , based on the following key assumption: n — 1 
local (on Q) first integrals can he expressed analytically for each vector field 
f{x, a), a € S. This assumption is imposed in the transient phase of a feedback 
system’s response, when the vector field is non-vanishing and local first inte- 
grals always exist, though analytical expressions for them may not be readily 
computable. 

If the assumption is met, then we can transform the hybrid system to a finite 
automaton. The control problem posed on the finite automaton is to synthesize 
a discrete supervisor, providing a switching rule between automaton locations, 
that minimizes a discrete cost function approximating the original cost function, 
for each initial discrete state. We provide a dynamic programming solution to 
this problem, with extra constraints to ensure non-Zenoness of the closed-loop 
trajectories. By imposing non-Zeno conditions on the synthesis we obtain piece- 
wise constant controls. The discrete value function depends on the discretizations 
of U and of 17 using the bisimulation. We quantify these discretizations by pa- 
rameters 5 and 5q, respectively. The main theoretical contribution is to show 
that as 5, Sq 0, the discrete value function converges to the unique viscosity 
solution of the Hamilton- Jacobi-Bellman (HJB) Equation. 

There is a similarity between our approach to optimal control and regular 
synthesis, introduced in [2], in the sense that both restrict the class of controls 
to a set that has some desired property and both use a finite partition to define 
switching behavior. Our work provides a constructive approach to obtain the 
cell decomposition by using a finite bisimulation, which further allows us to 
formulate the synthesis problem on its quotient system - a finite automaton. 
The idea of using a time abstract model formed by partitioning the continuous 
state space has been pursued in a number of papers recently. Lemmon, Antsaklis, 
Stiver and coworkers [10] use a partition of the state space to convert a hybrid 
model to a discrete event system (DES). This enables them to apply controller 
synthesis for DES’s to synthesize a supervisor. While our approach is related to 
this methodology, it differs in that we have explicit conditions for obtaining the 
partition. In [9] hybrid systems consisting of a linear time-invariant system and 
a discrete controller that has access to a quantized version of the linear system’s 
output is considered. This approach suffers from spurious solutions that must be 
trimmed from the automaton behavior. Hybrid optimal control problems have 
been studied in papers by Witsenhausen [11] and Branicky, Borkar, Mitter [3]. 
These studies concentrate on problems of well-posedness, necessary conditions, 
and existence of optimal solutions but do not provide algorithmic solutions. 

2 Optimal Control Problem 

Notation. !(•) is the indicator function. cl{A) denotes the closure of set A. 
II • II denotes the Euclidean norm. Let (7^(14”) and ^(11") denote the sets of 
continuously differentiable real- valued functions and smooth vector fields on IR", 
respectively. /i) denotes the trajectory of i = f{x, p) starting from xq and 

using control p(-). 
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Let U he & compact subset of M™, Q an open, bounded, connected subset 
of IR”, and 17/ a compact subset of 17. Define Um to be the set of meansurable 
functions mapping [0,T] to U. We define the minimum hitting time T : M” x 
Um by 



oo if {t \ fj,) € f2f } = 

min{t I 4>t{x,^) G 17/} otherwise. 



A control /t G Um specified on [0,T] is admissible for a; G 17 if G 17 for 

alH G [0, r]. The set of admissible controls for x is denoted Ux- Let 
TZ := { X G K” I 3/i G Ux- T{x, /i) < oo }. 



We consider the following optimal control problem. Given y G fi, 

rT{v,fi) 

minimize J{y,y,)= / L{x{t), y,{t))dt + h{x{T{y, y,))) 

Jo 

subject to x= f{x,y), a.e. t G [0,T(y,/i)] 

a;(0) = y 



( 2 ) 

( 3 ) 

( 4 ) 



among all admissible controls y G Uy. J : M” x Um — > K is the cost-to-go 
function, h : IR" ^ IR is the terminal east, and L : IR" x IR'" ^ IR is the 
instantaneous east. At T{y,y) the terminal cost h{x{T{y,y))) is incurred and 
the dynamics are stopped. The control objective is to reach 17/ from y G 17 with 
minimum cost. 

Assumption 2.1. 

(1) / : IR"xIR'" — > IR" satisfies \\f{x',u') — f{x,u)\\ < L/ [||a:'— a;|| + ||u'— u||] 
for some L/ > 0. Let M/ be the upper bound of ||/(a;,w)|| on 17 x {7. 

(2) L : IR" X IR'" — > IR satisfies \L{x', u') — L{x, w)| < Ll [||a:' — x|| + ||m' — u||] 
and 1 < |L(a;,u)| < Ml, x G fi, u GU , for some Ll,Ml > 0. 

(3) h : IR" — > IR satisfies \h{x') — h{x)\ < Lh\\x' — x\\ for some Lh > 0, and 
h{x) > 0 for all a; G 17. Let Mh be the upper bound of \h{x)\ on 17. 

The value function or optimal cost-to-go function V : IR" — > IR is given by 

V (y) = inf J (y, y) 



for y G 17 \ 17/, and by V{y) = h{y) for y G 17/. A control y is called e-optimal 
for X if J{x,y) < V{x) + e. It is well-known [7] that V satisfies the Hamilton- 
Jacobi- Bellman (HJB) equation 

-irf{L(x,u) + g/(a.,u)}=0 (5) 

at each point of TZ at which it is differentiable. The HJB equation is an infinites- 
imal version of the equivalent Dynamic Programming Principle (DPP) which 
says that 

V{x) =mfy,(zu,^^J*L{(j)s{x,y),y{s))ds + V{(pt{x, y))'\ , x G U \ Uf 
V{x) = h{x) 



X G 17/. 
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The subject of assiduous effort has been that the HJB equation may not have a 
solution. This gap in the theory was closed by the inception of the concept 
of viscosity solution [6], which can be shown to provide the unique solution of 
(5) without any differentiability assumption. In particular, a bounded uniformly 
continuous function V is called a viscosity solution of HJB provided, for each 
Ip G (7^(11”), the following hold: 

(i) a V — 'tp attains a local maximum at xq G M”, then 

~ ^(a^o)/(a^o,M)| < 0, 

(ii) ii V — p) attains a local minimum at xi G M”, then 

- ini^^L{xi,u) + ^{xi)f{xi,u)'^ > 0 . 



Assumption 2.2. For every e > 0 and x G TZ, there exists > 0 and 
an admissible piecewise constant e-optimal control fj, having at most 
discontinuities and such that (pt{x,^i) is transverse to di7f. 

The transversality assumption implies that the viscosity solution is contin- 
uous at the boundary of the target set, a result needed in proving uniform 
continuity of V . The finite switching assumption holds under mild assumptions 
such as Lipschitz continuity of the vector field and cost functions, and is based 
on approximating measurable functions by piecewise constant functions. 

3 Hybrid System 

The approach we propose for solving the continuous optimal control problem first 
requires a mapping to a hybrid system and, second, employs a bisimulation of 
the hybrid system to formulate a dynamic programming problem on the quotient 
system. In this section we define the hybrid optimal control problem. First, we 
discretize U by defining a finite set Us C U which has a mesh size 

S := sup min ||u — cr|| . 
ueu 

We define the hybrid automaton H := {S x S, D, Eh,G, R) with the 
following components. 

State set S x M” consists of the finite set A = U {cr/} of control locations 
and n continuous variables x G IR". cr/ is a terminal location when the 
continuous dynamics are stopped (in the same sense that the dynamics are 
“stopped” in the continuous optimal control problem). 

Events E = EsVJ {ct/} is a finite set of control event labels. 

Vector fields D : E ^ ^(11") is a function assigning an autonomous vector 
field to each location. We use the notation D{a) = fa- 
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Control switches Eh C S x S is a, set of control switches, e = (cr, cr') is a 
directed edge between a source location cr and a target location a' . If Eh{a) 
denotes the set of edges that can be enabled at a & E, then Eh{cr) := 
{{a, a') \ a' & E \ a} iov a & Ss and Eh(crf) = 0. Thus, from a source 
location not equal to af, there is an edge to every other location (but not 
itself), while location af has no outgoing edges. 

Enabling conditions G : Eh ^ {ge}e^Eh is a function assigning to each edge 
an enabling (or guard) condition g C M". We use the notation G{e) = ge- 
Reset conditions R : Eh ^ {re}e^Eh is a function assigning to each edge a 
reset condition, rg : M” ^ 2®^ , where we use the notation R{e) = r^- 

Semantics. A state is a pair (a,x), a G E and x G M". In location a G E^ the 
continuous state evolves according to the vector field f{x, a). In location a/, the 
vector field is i = f{x,fj,f) where gf is the (not necessarily constant) control of 
the terminal location. Trajectories of H evolve in steps of two types. A a-step is 

a binary relation -^C {E x IR") x {E x M”), and we write (cr, x) ^ (cr', x') iff (1) 

e = (cr, cr') G Eh, (2) X G ge, and (3) x' = re{x). The transition (a,x) {a',x') 

is taken at the first time in location a when the control event label is a' and 
a; G (/e for e = (a, o'). A t-step is a binary relation {E x M”) x (A x K"), 
and we write (a,x) (cr',x') iff (1) a = a', (2) at t = 0,a;' = x, and (3) for 

t > 0, x' = 4>t{x,a), where 4>t{x) = f {4>t{x , a) , a) . A hybrid control is a finite 
or infinite sequence of labels uj = ujiuj 2 ■ ■ ■, with uji G E U IR'*'. uji G IR''' is the 
duration of the f-step at step i. The set of hybrid controls is denoted S. A hybrid 
trajectory tt over w G 5 is a finite or infinite sequence tt : (ao,xo) ^ (cri,a;i) ^ 
(cT 2 ,a: 2 ) ^ . . . where (ai,Xi) G E x IR". Trajectory tt is accepted by H iff Vi, 
(cTj, Xi) (ai+i,Xi+i) is either a t-step or cr-step of El. Let tt be the trajectory 
(not necessarily accepted by El) starting at (cr, a;) G E x fl and defined over 
UJ G S. We say uj is admissible for (a,x) on interval [0,T] if (1) tt remains in 
A X 17 for t G [0,r], and (2) corresponding to w is a piecewise constant control 
Eojit) (with a finite number of discontinuities in finite time). Let be the 

set of admissible controls for (cr, a;). 

3.1 Hybrid Optimal Synthesis 

We want to synthesize enabling conditions so that for each y G TZ, the cost-to- 
go from y well-approximates the viscosity solution at y of HJB. This requires 
posing a hybrid optimal synthesis problem. We define a hybrid cost-to-go function 
Jh ■ E X IR" X 5 ^ IR as follows. For uj G 

Jh{{(J,x),uj) = J{x,g^^). 

The hybrid value function Vh '■ A x IR" ^ IR is 

Vh{{(t,x))= inf JH{{a,x),uj). 

^^‘5(0-, x) 

Hybrid optimal synthesis problem: 

Given H and 0 < < e^, synthesize ge, e G Eh, subject to: 
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1. ge = Of if e= {(J,(Jf), a G Ss- 

2. For each e G Eh, ge Q O. 

3. For alluj G S and (cr,x) G Ex O such that x)) < oo, is accepted 

by F[ if oj is admissible and -optimal for (cr,x). 

4- For all CO G S and {cr,x) G E x fl, 7T(a,x) is not accepted by H if either co is 
not admissible for (cr,x), oo is not -optimal for (cr,x), or Vff((a,x)) = oo. 

4 Construction of Bisimulation 

We propose to solve the hybrid optimal control problem using the bisimulation 
of H. In this section we define bisimulation and the quotient system that is 
obtained from it. 

Let A represent a t-step corresponding to some t G IR'*" . A bisimulation of FI 
is an equivalence relation ~C {Es x IR”) x {Es x IR") such that for all states 
Pi,P 2 G Es X IR”, if Pi ~ p 2 and a G U {A}, then if pi p[, there exists 
P 2 such that p 2 P 2 and p[ p' 2 . If cx is finite, the quotient system is a finite 
automaton. 

Since the dynamics are restricted to the set O, the set of interesting equiva- 
lence classes of denoted Q, are those that intersect Esxcl{0). For each q G Q 
we define a distinguished point (cr, G q. We associate q with its distinguished 
point by the notation q = [(cr, ^)]. It is now possible to define the enabling and 
reset conditions of F[ in terms of Q. In particular, the enabling conditions of F[ 
are synthesized as subsets of Q while the reset conditions are defined as follows. 
For e = (cr, cr') 

re{x) = {y \ 3^.[(cr,a;)] = [(cr,^)] A [(cr',^)] = [{a',y)] }. (6) 

That is, re(x) is the projection to IR” of the set of equivalence classes [(tr', y)] such 
that the projection to IR" of [{cr',y)] and [{a,x)] have nonempty intersection. 
This definition in effect gives an over-approximation of the identity map in terms 
of the equivalence classes of ~ and will introduce non-determinacy in the finite 
automaton. Notice also that (6) encodes information about the bisimulation 
in H. This sequence of steps is not typical; it is characteristic of our synthesis 
procedure. We define a mesh size on Q hy Sq = maxggg sup(^_ 2 ,) (o- — y||}. 
Finally, for each q = [(cr, ^)] G Q we associate the duration Tq, the maximum 
time to traverse q using constant control a. That is, Tq = sup(^ t \ y = 

(j>t{x,a) }. 

Geometric construction. We give a brief review of the method developed in 
[4] for obtaining bisimulations. We require the following (related) assumptions 
on the vector fields on cl{0). 

Assumption 4.1. 

(1) n — 1 first integrals can be defined analytically on O for each f{x,a), 
cr G Es- 

(2) There exists m/ > 0 such that ||/(a;, m)|| > m/ for all x G d(0), uGU. 
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A bisimulation of Ss x IR" is constructed using a set of simple, co-dimension 
one tangential foliations with associated submersions 7 f (a;) = yf,i = 1,. . . ,n—l 
and a simple co-dimension one transversal foliation with submersion 7 ^ = y^, 
such that (j/i , . . . , y’^) form a set of euclidean coordinates for each a G Eg. We 
discretize the foliations by selecting a finite set of leaves. Fix k G and let 
A = ^. Define 

Ck = {0,±A,±2A,...,±l}. (7) 

Each yf = c for c G Ck, i = 1, . . . , n defines a hyperplane denoted and a 
submanifold The collection of submanifolds for cr G is 

Wfe ={ I cGCfc.zG n} }. (8) 

17 \ yVk union of disjoint open sets = {V^}. We define the 

equivalence relation ~ on A 5 x IR” as follows: {a, x) ~ {a' , x') iff (1) a = a' and 
(2) a; G VF iff x' G W, and a; G U iff x' G U, for all W G and V G V^. 



5 Discrete Problem 



In this section we transform the hybrid optimal control problem to a dynamic 
programming problem on a non-deterministic finite automaton, for which an 
algorithmic solution may be found. Consider the class of non-deterministic au- 
tomata with cost structure represented by the tuple A = (Q, Es, E, obs, Qf,L,h). 
Q is the state set, as above, and Es is the set of control labels as before. 
obs : E ^ Es is a, map that assigns a control label to each edge and is given by 
obs{e) = a', where e = {q, q'), q = [(cr, ^)] and q' = [(cr',^')]. Qf is the target set 
given by the over-approximation of f2f,Qf = {q€Q \ 3xG 17/ . (a,x) Gg}. 

E C Q X Q is the transition relation encoding t-steps and cr-steps of H. 
A will be used to synthesize ge of H , so E includes all possible edges between 
locations. The synthesis procedure on A will involve trimming undesirable edges. 
Thus, {q,q') G E, where q,q' & Q, q = [(o’, C)] and q' = [(u',^')] if either (a) 
a = a', there exists a: G 17 such that (a, x) G Q, and there exists r > 0 such that 
Vt G [0, t], (ct, a)) G q and (a, c/>r+£(x, a)) G q' for arbitrarily small e > 0, or 
(b) (J = cr', there exists a; G 17 such that (cr, x) G Q, and there exists t > 0 such 
that Vt G [0 ,t), (a, 4>t(x, a)) G q and (a, 4>r(x, (t)) G q', or (c) a ^ a' and there 
exists a; G 17 such that (cr, a:) G Q and (cr',a;) G q' . Cases (a) and (b) say that 
from a point in q, q' is the first state (different from q) reached after following 
the flow of f{x, a) for some time. Case (c) says that an edge exists between q 
and g' if their projections to IR” have non-empty intersection. 

Let e = (g, g') with g = [(cr, ^)] and g' = [(cr',^')[. L : E ^ M is the discrete 
instantaneous cost given by 



L(e) 



/ rqL{^, cr) if cr = cr' 
1^ 0 if cr yf cr'. 



(9) 



This definition reflects that no cost is incurred for control switches. : Q — > IR 
is the discrete terminal cost given by 



h{q) := /i(C). 
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The domain of h can be extended to fi, with a slight abuse of notation, by 
h{x) := h{q) where q = argming/{||a; - ^'|| | q' = [(cr', ?')]}• 



5.1 Semantics 

A transition or step of A from q = [(cr, ^)] G Q to q' = G Q with 

observation cr' G Ss is denoted q ^ q' . If a ^ a' the transition is referred to as 
a control switch; otherwise, it is referred to as a time step. If E{q) is the set of 
edges that can be enabled from q G Q, then for a G Ss, 

Ea{q) = {e G E{q) \ obs{e) = a}. 

If \Ea-{q)\ > 1, then we say that e G E^{q) is unobservable in the sense that 
when control event cr is issued, it is unknown which edge among E„{q) is taken. 
If cr = cr', then \Ea{q)\ = 1, by the uniqueness of solutions of ODE’s and by the 
definition of bisimulation. 

A control policy c : Q ^ Es is a, map assigning a control event to each 
state; c{q) = cr is the control event issued when the state is at q. A trajectory 
7 T of A over c is a sequence tt = go ^ 9i ^ <72 ^ ■ ■ ■, qi G Q. A trajectory 
is non-Zeno if between any two non-zero duration time steps there are a finite 
number of control switches and zero duration time steps. Let EIc{q) be the set of 
trajectories starting at g and applying control policy c, and let EIc{q) be the set 
of trajectories starting at g, applying control policy c, and eventually reaching 
Qf. If for every g G Q, tt G EIc{q) is non-Zeno then we say c is an admissible 
control policy. The set of all admissible control policies for A is denoted C. 

A control policy c is said to have a loop if A has a trajectory go gi 

. . . qm = qo, qi & Q- A control policy has a Zeno loop if it has a loop 

made up of control switches and/or zero duration time steps only. One can show 
that a control policy is admissible iff it has no Zeno loops. 

5.2 Dynamic Programming 

In this section we formulate the dynamic programming problem on A. This 
involves defining a cost-to-go function and a value function that minimizes it 
over control policies suitable for non-deterministic automata. 

Suppose 7T = go ^ gi ^ . . . ^ gw-i ^ g^v G 77, where qi = [(<Ti,^i)] 
and 7T takes the sequence of edges eie 2 ...eAr. We define a discrete cost-to-go 
J : Q X C — > IR by 



= I Hej) + HqnJ} if n^iq) = ndq) 

I oo otherwise 

where = minjj > 0 | g^ G Q/}. We take the maximum over lJc{q) because of 
the non-determinacy of A: it is uncertain which among the (multiple) trajectories 
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allowed by c will be taken so we must assume the worst-case situation. The 
discrete value function V : Q ^ IR is 

V (q) = min J {q, c) 

for q G Q\Qf and V{q) = h{q) for q G Qj. We show in Proposition 1 that V 
satisfies a DPP that takes into account the non-determinacy of A and ensures 
that optimal control policies are admissible. This DPP describes the accumula- 
tion of cost over one step to be the worst case cost among edges that have the 
same label. Let Aq be the set of control assignments c{q) G Ss a,t q such that c 
is admissible. 

Proposition 1. V satisfies 

V{q)= inin | max {L{e) + V{q')}\ , qGQ\Qf (10) 

c(g)eAg fe=(q,q')eEc(q)(g) J 

V(q) = h(q), q&Qf- (H) 

5.3 Synthesis of 

The synthesis of enabling conditions or controller synthesis is typically a post- 
processing step of a backward reachability analysis (see, for example, [12]). This 
situation prevails here as well: equations (lO)-(ll) describe a backward analysis 
to construct an optimal policy c G C. Once c is known the enabling conditions 
of H are extracted as follows. 

Consider each e = G E oi H with a a'. There are two cases. If 

cr' (7/ then = {x \ {a,x) G q^q G Q A c{q) = cr'}. That is, if the control 
policy designates switching from q G Q with label a to q' G Q with label a', 
then the corresponding enabling condition in H includes the projection to IR" 
of q. The second case when cr' = ct/ is for edges going to the terminal location 
of H. Then ge = {x \ (cr,x) G q,q G Qf} . 

6 Main Result 

We will prove that V converges to V, the viscosity solution of the HJB equation, 
as Sq,S — > 0. The proof will be carried out in three steps. In the first step we 
consider restricting the set of controls to piecewise constant functions, whose 
constant intervals are a function of the state. In the second step we introduce 
the discrete approximations of L and h. In the last step we introduce the discrete 
states Q and consider the non-determinacy of A. 

In the sequel we make use of a filtration of control sets = Es^ correspond- 
ing to a sequence Sk ^ 0 &s k ^ oo, in such a manner that Ek C Ek+i - Consider- 
ing (8), we define a filtration of families of submanifolds such that C 
for each a G Ek- 
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Step 1: piecewise constant controls. 

In the first step we define a class of piecewise constant functions that depend 
on the state and show that the value function which minimizes the cost-to-go over 
this class converges to the viscosity solution of HJB as <5^ ^ 0. The techniques 
of this step are based on those in Bardi and Capuzzo-Dolcetta [1] and are related 
to those in [5]. 

We consider the optimal control problem (2)-(4) when the set of admissible 
controls is U\, piecewise constant functions consisting of finite sequences of con- 
trol labels a G Sk and each cr is applied for a time r(cr, x). Let (a, x) G q for some 
q G Q and define r(cr, x) to be the minimum of the time it takes the trajectory 
starting at x and using control a G Sk to reach (ta) df2f, and (tb) some x' such 
that (a, x') ^ g. If a trajectory is at Xi at the start of the {i + I)th step, then the 
control (Ji+i is applied for time := T{ai+i,Xi) and Xi+i = (a^ij CTi+i). 

Let 

TZi. ■.= { X G M" \ 3^ G . T{x, fx) < oo }. 

We define the cost-to-go function Jl : H x Ul ^ M as follows. For x G 17 and 
fj, = (Ti(T 2 ■ . ■ GUI, if T{x, /x) < oo then 

Jl{x,^i) = 2^ / L{(j>s{xj-i,crj),aj)ds + h{xN) 

J=i 

where N = min{j > 0 | G dflf}. = oo, otherwise. We define the 

value function : IR" ^ IR as follows. For x G 17 \ I7y, 

Vk{x)= inf Jl{x,n) (12) 



and for x G 17/, V^{x) = h{x). 

{Vj^} forms a family of equibounded, locally equicontinuous functions. It can 
then be shown that, along some subsequence kn, converges to a continuous 
function 14. Moreover, the following holds: 

Proposition 2. 14 is the unique viscosity solution of HJB. 

Step 2: approximate cost functions. 

In this step we keep the semantics on piecewise constant controls of Step 1 
but replace cost functions L and h by approximations and h. We define the 
cost-to-go function J| : 17 x ^ IR as follows. First, we define an approximate 
instantaneous cost : 17 x L'fc ^ IR given by 

L^(x,cr) := L{q) 

where (cr, x) G q. For x G (7 and fi = a±a 2 ■ ■ ■ if T(x, /x) < cx) then 

N 

= + h{xN) 

i=i 



(13) 
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where N = min{j > 0 | G 917/ }. We define a value function : M” ^ IR as 
follows. For X G n \ 17/, 

V^(x)= M (14) 

ti&Jl 

and for x G 17/, V^{x) = h{x). For a; G 17 such that V^{x) < oo, V)? satisfies the 
DPP V^{x) + V^{(l)^(^^^^){x,a))}. 

Remark 6.1. 

For each x G and e > 0 there exists m G .S'*’ and /i G such that /t 

is an e-optimal control for x w.r.t. satisfying Assumptions 2.2. This follows 
from Assumptions 2.2, V^{x) > V{x), and the fact that we can well-approximate 
an e-optimal control for P by a control in for large enough m. 

Proposition 3. Let ko G , x G and /i G be an e-optimal eontrol 

for X. Then \J^{x, /t) — J^(a:, /t)| — > 0 as k ^ oo. 



Proof. First, we require two facts which are stated without proof, for brevity. 
Fact 1. If 5k < then for all q G Q, 



Tq < 



5k 



rrif — Lf5k 



(15) 



For the next fact, we require a definition, let Ck be as in (7) and 7 )^ the 
transversal foliation of a; = f{x,a). For cr G Sk, define the region in IR" 



:={xG ini) ^(c) I cG Cfe }. 



Fact 2. Let x,x' G M)f for some c G Ck and a G Sk- Let t,t' be times such 
that 4>r{x, a), (j)ri {x' , a) G Then |r — r'| < c^rdk for some c.y > 0. 

Now we have 



Jk{x,ti) - Jl{x,p) < 



N 



pT{aj,Xj--i) 



sL 

j=l 
N 

-Hxn) 

i=i 



L{4>sixj-i,aj),aj)ds h{xN) 



where (xj-i,aj) G qj-i and qj-i = [(^/_i, ct/)]. There exists such that 

h{xN) = /i(Cw) and ||a;Ar — ^Ar|| < 5k. Also, using the Mean Value Theorem, 
there exists t with x = <f>i{xj-i, Uj) and ||i — Ci-i|| ^ such that 



N 

Jk{x,T) - Jk{x,T)\ < 

i=i 

-I- |li(a;Ar) - h{xN)\ 

N N 

< “ ^(f^a.a;/-i)]T(i,cr/) -f La4- 

i=i i=i 
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Using Fact 1 the first term on the r.h.s. decreases linearly as 6k- Call the second 
term on the r.h.s. “B”. Splitting B into sums over control switches and time 
steps, we have 

N 

B < - T{aj,Xj-i)]l{aj = aj-i) 

i=2 

N 

+ Ml - T{a + aj-i) 

i=i 

N N 

<MlY^ Cj-lTg^_,Sk +MlY^ 7^ M-l) 

i=2 i=i 

for some Cj-i G M. In the second line we used Fact 2 and the fact that Tg^_j > 
T{aj,Xj-i). Using Fact 1 the first term on the r.h.s. decreases linearly as 6k- 
The second term on the r.h.s. goes to zero since /i has a fixed number of control 
switches for all fc > fcg. □ 

Step 3: discrete states and non-determinacy. 

We define Vk{x) := mincrei;fc{ Vk{q) \ {cr,x) & q }- Also let TZk = {x G 
C I Vk{x) < oo} and Iz = UkiZ-k- 
Remark 6.2. 

(a) By Remark 6.1 and V^{x) < V^{x)^ for each x G ^kTZ\ e > 0 there 
exists rrie G and /i G such that /r is an e-optimal control for x w.r.t. 

satisfying Assumptions 2.2. 

(b) it C UfcT?.^, but the converse is not true, in general. 

(c) If /i is an e-optimal control for x w.r.t. V^, then we can assume /i) does 
not self-intersect, for if it did we can find p,, also e-optimal, which eliminates 
loops in 4>t{x, /i). 

(d) ||a; — y|| ^ 0 as fc ^ oo for all y G re{x) and all edges e of Hk, the hybrid 
automaton defined using Sk and Ck given in (7). 

Proposition 4. For all x G It, \Vk{x) — V^{x)\ ^0 as k ^ oo- 

Proof- Fix e > 0 and x G TZ- By Remark 6.2(a) there exists m-e > 0 and an 
e-optimal control /j G for x- Let us denote /i as an open loop control fi = 
((cTi, Ti ),..., (<T AT, Tat)), where Ti is the time Ui is applied. If c is a policy derived 
using 6k and Ck, for k > rrie, then 0 < Vk{q) — Vji{x) < Jk{q, c) — J^{x, y) + e, 
where q = [{ai,x)\- If we can show there exists k > me such that for k > k, 
there exists a policy c such that Jk{q,c) — Jl{x,y) < e and using the fact that 
l^fc(<z) — Ufe(a;)| ^ 0 as fc ^ oo, then the result follows. 

Consider the set Fk of (discontinuous) trajectories <j)t{x,y) where p G Ul 
is denoted ((cri, fi, . . . , (ctat, tat)) . Also x~ = (j>f^{xj-i, aj) and Xj G Te{x~), 
where e = ((jj,(jj+i) is an edge of Hk, defined in Remark 6.2(d). We can find 
k\ > me such that, by Remark 6.2(d) and the transversality of <it{x, y) with the 
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submanifolds where it switches controls and with f2f, there exists p, G Ul such 
that G 'I'k switches controls on the same (transversal) submanifolds and 

reaches 17/. Let L'^{xj-i,<Tj) + h{xN)- We observe that for S 

if'fc and ^ gU^ , \W‘^{(f)) — W“^{(j)')\ ^ 0 as fc ^ oo, using Lipschitz continuity of 

k\ 

L and h, Remark 6.2(d), and the fact that /i is fixed for all k> k\. Notice that 
J^{x, y) = for some <p G 'I'k- We can define the control policy c such that 

automaton A accepts the time abstract trajectory starting at q corresponding 
to each trajectory of 'I'k and with all other control assignments of c as time 
steps, c is admissible because otherwise some <j>' G 't'k would have a Zeno loop. 
Since (j>' approaches 4>t{x,fj,) as fc ^ oo, this would imply 4>t{x,fj,) has a loop, 
contradicting Remark 6.2(c). Now we observe that J{q,c) = max,^g,j^j, := 

Thus, Jk{q,c) - J^{x,fx) < \W^{(j)) - ^ 0 as /c ^ oo. □ 



Theorem 1. For all x GTZ, Vk{x) ^ V{x) as k ^ oo. 

7 Conclusion 

In this paper we have developed a methodology for the synthesis of optimal 
controls based on hybrid systems and bisimulations. The idea is to translate 
an optimal control problem to a switching problem on a hybrid system whose 
locations describe the dynamics when the control is constant. When the vector 
fields for each location of the hybrid automaton have local first integrals which 
can be expressed analytically we are able to define a finite bisimulation using the 
approach of [4]. From the finite bisimulation we obtain a (time abstract) finite 
automaton upon which a dynamic programming problem can be formulated that 
can be solved efficiently. 

We are presently working on three topics that will enhance considerably the 
significance of our work: 

— The dynamic programming problem is equivalent to a shortest path problem 
on a non-deterministic graph. We are in the process of carrying through the 
implementation issues to obtain an algorithmic solution. 

— Throughout the paper we have assumed that, once the bisimulation is ex- 
pressed using first integrals, the corresponding finite automaton can be con- 
structed directly. In fact, this task is not so straightforward. We are working 
on the automatic generation of finite automata that give time abstract be- 
havior of vector fields. 

— If it is not possible to obtain a finite bisimulation, one may still be able 
to construct a finite automaton that approximates the continuous and dis- 
crete behavior of the hybrid system. But this automaton will have non- 
deterministic behavior that results in spurious solutions, not corresponding 
to the true dynamics of the hybrid system. We are working on a procedure 
to eliminate these spurious solutions. 
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Abstract. In this article, we show how a behavior based control system 
for autonomous robots can be modeled as a hybrid automaton, where 
each node corresponds to a distinct robot behavior. This type of con- 
struction gives rise to chattering executions, but we show how regular- 
ized automata suggest a solution to this problem. We also discuss some 
design and implementation issues. 



1 Introduction 

For mobile, autonomous robots the ability to function in, and interact with 
a dynamic, changing environment is of key importance. A successful way of 
structuring the control system in order to deal with this problem is within a 
behavior based control architecture [3]. The main idea is to identify different 
controllers, responses to sensory inputs, with desired robot behaviors. A behavior 
could, for instance, be obstacle avoidance in which sonar information about a 
close obstacle should result in a movement away from that obstacle. This way of 
structuring the control system into separate behaviors, dedicated to performing 
certain tasks such as avoid obstacles or traverse doors, has turned out to be a 
successful design. It has the major advantage that it makes the system modular, 
which both simplifies the design process as well as offers a possibility to add new 
behaviors to the system without causing any major increase in complexity. 

The suggested outputs from the different, concurrently active behaviors are 
fused together according to some action coordination rule, and this makes it 
easy to stress such questions as safety explicitly, since, for example, an avoidance 
behavior can just be given higher priority than a reach target behavior. 

However, within this framework, a number of design issues still need to be 
addressed. Those range from questions concerning the design of the individual 
behaviors to action coordination issues [5]. For instance, given a reactive obstacle 
avoidance behavior, modeled as a repulsive field surrounding the obstacle, how 
should an approach target behavior be designed so that it takes advantage of 
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the fact that it is going to run in parallel with an obstacle avoidance behavior? 
Furthermore, how should these behaviors be combined? 

What will be investigated in this article is how a behavior based system can 
be modeled as a hybrid automaton with each of the discrete nodes corresponding 
to a distinct behavior. If the system where to be described by such an automaton 
it would hopefully help us understand and explain some of the so called emergent 
phenomena that complex robotics systems can give rise to. We will furthermore 
see that questions concerning safety and optimality can be addressed nicely 
within this framework. 

The outline of the article is as follows: First, in Section 2, we discuss some 
of the properties of a behavior based robotics system, and we show how this 
can be modeled as a hybrid automaton. Some regularization techniques are then 
exploited in order to get rid of potential chattering in the automaton. In the next 
section, some control design issues are discussed, and we describe a heuristic 
method for constructing behaviors that are safe at the same time as they are 
close to optimal with respect to a given performance evaluation functional. We 
conclude, in Section 4, with a brief discussion about a proposed, systematic 
strategy for implementing the hybrid automata. 



2 Behavior Based Robotics 

As already mentioned, for autonomous robots operating in a partially unknown, 
dynamic environment a successful way of structuring the controllers is within 
a behavior based framework [3], [10]. Different robot behaviors are identified, 
e.g. obstacle avoidance or reach target, and their functionality is defined by a 
tight mapping from sensory data to a desired action. Typically, in a so called 
reactive behavior based system, no representation of the world is contained in 
this mapping, while a deliberative system exploits planning or world models in 
the control loops. 

The desired output actions are then normally fused together by an arbitration 
mechanism, as seen in Figure 1, where a wide-spread solution to the action fusion 
problem, used for example in the schema theoretical paradigm, is to represent the 
goals, targets and obstacles by weighted attractive or repulsive potential fields, 
resulting in weighted, desired orientation vectors. The action coordination is then 
simply done using vector summation. This way of letting behaviors be active 
simultaneously is desirable in many situations. For instance, while approaching 
a target an obstacle avoidance behavior has to be active for safety reasons while 
the performance is improved if the robot tries to approach the goal at the same 
time as it is avoiding obstacles. This calls for a fused, coordinated control scheme 
[2], [3]. 



2.1 Obstacle Negotiation 

The specific problem that will be investigated in this article is how to move a 
robot between two points. This point-to-point motion should be done so that 
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Fig. 1. Block diagram of the behavior based control architecture. 



the detection of an obstacle results in a repulsive potential field, acting on the 
platform when the robot is closer to the obstacle than a desired safety distance, 
do A, where the subscript stands for obstacle avoidance. This behavior is an 
example of a so called reactive obstacle avoidance behavior. The word reactive, 
a commonly used one in the robotics community, is used here since the behavior 
can be thought of as a reflex. When the robot moves too close to an obstacle, 
it is forced to change the motion in order to avoid hitting the obstacle. This is 
a reasonable safety strategy since the robot may be moving around in a highly 
unstructured world, where the occurrence of unpredicted, or unmodeled obstacles 
is very likely. 

We now assume that we have direct access to the robot’s longitudinal velocity, 
u, at the same time as the heading of the robot, 4>, can be controlled directly as 

4> = uj. 



Furthermore, if the sonars on the robot, with center of gravity at {x, y) and 
heading <j), detect a point-obstacle at {xob, Uob) that is closer to the robot than 
do A, the reactive control response will be given by a vector field acting on the 
robot as 

ui = CoAWoA{d){4> - 4>), ( 1 ) 

d = i/(a; - Xob)'^ + {y- VobY, 



where 

WoA{d) 



^ if d < do A 
0 if d > do A, 



and 



(f) = 'K + atan2(j/oh - y, Xob ~ x), 



( 2 ) 



as seen in Figure 2. Here, Coa is just a constant weight, and do a is the fixed 
distance from the obstacle where the behavior becomes active. 

Since a real, extended obstacle cannot be considered to be a point, in the 
actual implementation of the avoidance behavior, the desired heading needs to 
be calculated as the orientation of the sum of the weighted vectors that each 
individual sonar reading contributes with. For a Nomad 200, that is going to be 
our experimental platform, this corresponds to taking the sum over 16 elements 
since the Nomad is equipped with 16 ultrasonic sensors. 

A standard kinematic model of the mobile robot [1] gives that 



X = V cos 0 
y = V sin 0, 



(3) 
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( ^ob’ Yob ) 




(x,y) 



Fig. 2. The general idea behind a repulsive obstacle avoidance behavior. 



and if we set z = {x,y,(j)), we let i = foA{z) denote the full state, closed-loop 
obstacle avoidance behavior where we initially let v be constant. 

2.2 Hybrid Automata 

When adding a goal attraction behavior, defined in the same way as the obstacle 
avoidance behavior except that we now have an attractive instead of a repulsive 
field, we get two different possible hybrid automata for describing the situation. 
This depends on whether the two behaviors are active simultaneously or not, 
as seen in Figure 3. If one chooses to work with fused, concurrently active be- 
haviors, then different controllers affect the system simultaneously, resulting in 
a smooth overall performance [11]. But in that case, however, the system does 
not correspond to an automaton where each node represents a single behavior. 
This would make the automata approach meaningless since we would then just 
“hide” all of the difficulties that the complex control system gives rise to in the 
individual nodes of the automaton. 

On the other hand, the other possible solution to the coordination problem, 
corresponding to hard switches between the different behaviors, has the major 
disadvantage that it both affects the performance in a negative way, not allowing 
for the smooth performance that fused behaviors produce, and that it increases 
the risk of introducing chattering into the system. Therefore our idea is to impose 
hard switches on the behavior based system in such a way that we can model 




(a) Fused behaviors 



(b) Hard switches 



Fig. 3. The two possible goal attraction and obstacle avoidance automata. Here, 
doA is the fixed distance from the obstacle where the obstacle avoidance behavior 
becomes active. 
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each behavior as a node in an automaton, at the same time as we want to 
avoid the negative, chattering effects that such an approach could potentially 
give rise to. This will be done by adding nodes to the automaton as a way of 
regularizing it, and in what follows we will show that even though we introduce 
hard switches, the performance is not affected much when using a regularized 
automaton instead of fused behaviors. In other words, what we want to do is 
to remove some of the so called Zeno^ properties of the system. What this 
corresponds to is a hybrid system that exhibits an infinite number of discrete 
transitions in finite time. 

Even though the main focus in this article is not going to be on hybrid 
automata theory, we need to include some initial definitions. This is necessary 
in order to be able to state what we mean by a Zeno hybrid automaton as well 
as to capture the hybrid aspects of a behavior based robotic system. 

The following brief definitions are based on [6], [8], [14]. 

Definition 1 (Hybrid Automaton). A hybrid automaton is considered to he 
a collection {Q,X,I,f,E) where Q and X are sets of discrete and continuous 
variables respectively. I is a set of initial states, while f describes the continuous 
and E the discrete evolution of the states. 

A discrete state combined with the continuous dynamics connected to that state 
will be referred to as a node in the automaton. The general idea behind this 
construction can be seen in Figure 4. 



(x,q,q)eE 







(x.q ’,q) ( 

Fig. 4. The basic structure of a hybrid automaton. 



Definition 2 (Hybrid Time Trajectory). A hybrid time trajectory t is a 
finite or infinite sequence of intervals of the real line, r = {h}, t G N, satisfying 
the following conditions: 

— li is closed, unless t is a finite sequence and li is the last interval in which 
case it can he right open. 

— Let li = [Ti,r']. Then for all i, Ti < t[ and for i > 0, Ti = 

This should be interpreted as the times at which we arrive (t^) and leave (r() a 
specific node in the automaton. 

^ The name Zeno refers to the philosopher Zeno of Elea (500-400 B.C.), whose major 
work consisted of a number of famous paradoxes. They were designed to explain his 
view that the ideas of motion and evolving time lead to contradictions. An example 
is Zeno’s Second Paradox of Motion, in which Achilles is racing against a tortoise. 
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Note that hybrid time trajectories can extend to infinity if t is an infinite 
sequence or if it is a finite sequence ending with an interval of the form [tat, oo). 

Definition 3 (Execution). An execution x of a hybrid automaton H is a col- 
lection X = {T,q,x), satisfying 

— Initial Condition: (g(ro), x(to)) € /. 

— Discrete Evolution: (a:(r'_]^), g(Ti)) G E, for all i. 

— Continuous Evolution: for all i with Ti < t[, x and q are continuous over 

and for all t G we have ^x{t) = f [q{t) , x{t)) . 

Furthermore, an execution y = (r, q, x) is called infinite, if r is an infinite se- 
quence, or ~ e) = oo. We use Hiqo^xo) to denote the set of all infinite 

executions of H with initial condition (qo,xo) G I. An execution is admissible 
if ~ d) = oo, and it is Zeno if it is infinite but not admissible. For a 

Zeno execution y = (t, q, x) we define the Zeno time as Too = ~ e ) < oo. 

What this means is that the hybrid system makes an infinite number of discrete 
transitions in finite time, [to,Too], and we finally state the following definition. 

Definition 4 (Zeno Hybrid Automaton). A hybrid automaton H is called 
Zeno, if there exists (qo,xo) G I such that H(qg^xo) contains a Zeno execution. 

2.3 Regularization 

It is clear that a Zeno hybrid automaton has the undesirable property that it 
blocks time. For the type of automata that we will encounter here, the infinite 
number of discrete transitions, made in finite time, is caused by the fact that the 
underlying system that the automaton tries to model is a switched system that 
exhibits sliding in the sense of Filippov [7]. They thus form a special class of Zeno 
hybrid automata since they, in theory, make an infinite number of transitions in 
zero time.^ The underlying, switched systems have continuous flows that point 
toward the switching surface, resulting in a new, induced flow on that surface. In 
these cases, the automaton can be regularized by the introduction of a new node 
with the continuous flow given by the Filippov solution [8], [15]. The general idea 
behind this construction can be seen in Figure 5. 

If we now assume that Co a in (1) is large enough so that the heading of the 
robot can be considered to be more or less instantaneously driven to its desired 
configuration, the hybrid automaton in Figure 3(b) can admit Zeno executions. 
This obvious fact is best illustrated by Figure 6, where the extra node that needs 
to be added in order to regularize the automaton can easily be identified as well. 
The extra node is just a node containing the sliding dynamics that is defined on 
the boundary between the two behaviors. 

When an obstacle is closer to the robot than do a, the obstacle avoidance 
behavior becomes active. Since the repulsive potential field from that behavior 

^ The other class of Zeno automata has a slightly more complex dynamics. Here the 
automaton changes nodes faster and faster, with the jump times converging to the 
Zeno time, Tcx> [8]. 
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a ^0 




tion. ularized automaton. 

Fig. 5. Regularization of a Filippov type Zeno hybrid automaton. 



is orthogonal to the surface on which the behavior becomes active, the sliding 
solution is just 

fs = afoA + (1 - a)fGA, 

where GA stands for goal attraction, and a € [0, 1] is chosen so that fs -L foA- 
Adding this type of information about the different behaviors makes it possible to 
generate the extra node in the automaton automatically. It furthermore suggests 
that our method would scale when more that two behaviors affect the motion of 
the robot, as long as an automatic procedure for designing the sliding solutions 
could be identified for the new behaviors as well.^ 



or 

Robot 




Obstacle 



->o 

Goal 



Fig. 6. Goal attraction together with obstacle avoidance results in a Filippov 
type Zeno automaton. The grey region around the obstacle corresponds to the 
region where obstacle avoidance is active. The arrows correspond to the different 
vector fields that are acting on the robot. 



The assumption about instantaneous heading control is obviously a simpli- 
fication but it still gives a model that is rich enough to capture the, from our 
point of view, relevant phenomena. In fact, in real life we have a possibility of 
chattering that here reveals itself as a Zeno execution. 

® This typically depends on whether we have access to a geometric description of the 
switching surface or not. 
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(a) No- (b) Fused be- 
mad haviors 

200 



(c) Hard 

switches 



(d) Regularized 
hybrid automa- 
ton 



Fig. 7. Simulation of b) fused behaviors, c) hard switches, and d) a regularized 
automaton on the Nomad simulator, the Nserver. 



The regularized point-to-point automaton was implemented and tested on a 
Nomad 200 mobile robot. In Figure 7, the results from running the system on the 
Nserver, the Nomad simulation package, can be seen. In (7b) fused behaviors 
are displayed, resulting in a smooth movement around the obstacle, while the 
chattering solution in (7c) corresponds to hard switches. The reason why we do 
not have sliding in this case is due to the part of the dynamics of the robot that 
was ignored in the analysis. It is still clear that from a performance perspective, 
(7c) is an unsuccessful design. In (7d) the result from using a regularized au- 
tomaton can be seen, and even though we only have one behavior active at a 
time, the performance is satisfactory. 

3 Controller Design 

Given the reactive obstacle avoidance behavior from the previous section, the 
main question that we want to address here is: How do we construct an appro- 
priate approach target behavior? Obviously, we can do better than to just use 
an attractive potential field, and it will turn out that our automata approach 
allows us to explicitly deal with safety and optimality. 

What we want to do is to produce a robot behavior that satisfies the safety 
specifications at the same time as the solution is close to optimal with respect 
to a given performance evaluation functional, and a first formulation, inspired 
by [19], of what we want to accomplish could be the following. If we let our 
admissible controls he u and define a safety functional 

Js{u) = min {(a;(t) - XobY + (2/(^) “ VobY] , (4) 

where the dependence on the control, u, is given implicitly by the controlled 
system dynamics from the previous section. The set of controls, Us{C), that 
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make the robot move at least a distance C away from the obstacle, can thus be 
defined as 

U,{C) = {u€U : Js{u)>C}. (5) 

It should be mentioned that both J7s and Us depend on the robot’s initial posi- 
tion, but for the sake of notational simplicity we leave that out from the defini- 
tions. 

The next step is to define another cost functional that penalizes high cur- 
vature of the chosen path. This is a reasonable performance criterion since it 
penalizes paths that make the robot move in sudden, abrupt ways. Furthermore, 
this smoothness objective gives a trajectory that a robot has good chances of 
following when it is governed by physical limits on what signals the actuators 
can actually track. In some other situations, such as when a mobile manipulator 
is asked to carry a cup of coffee, the smoothness of the curve is absolutely crucial 
and is obviously of key importance to a successful, “non-spilling” execution of 
the task. 

The idea now is to choose the control candidates for minimizing this new 
performance functional from the set of safe controls, Ug{Cs), where Cg is our 
preferred safety margin. 

Unfortunately it turns out that this is a very hard problem to solve numeri- 
cally (not to mention analytically) [19], which implies that in this formulation, it 
is not suitable for situations where on-line computations are necessary. However, 
the underlying approach could suggest a way for producing a solution to the ob- 
stacle negotiation problem that is both safe, computationally feasible, and makes 
the system behave in a satisfactory way with respect to keeping the curvature 
of the produced path small. 

The main idea is that instead of focusing on the hard optimal control problem, 
we should concentrate on just producing optimal (or close to optimal) geometric 
trajectories that lead around the obstacle. This way we do not have to deal 
with the actual kinematics of the robot in the optimization formulation. Instead 
we add the kinematics when we track the produced path. This means that we 
cannot be sure that we actually find the optimal controller, but rather that we 
find one that is reasonably close to the optimal one as long as we have a good 
enough trajectory tracker. 

The desired overall behavior that these heuristics give rise to (under the 
assumption of perfect tracking), together with the corresponding automaton, is 
depicted in Figure 8. 

3.1 Path Planning 

One first observation is that for a path produced by a scalar function yd = f{xd), 
the curvature is given by 

/ ^ f'ixd) 

where the subscript d stands for the desired robot position. 
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(a) 



(b) 



Fig. 8. In the left figure, an optimal path is planned and followed by an Ap- 
proach Target behavior until an obstacle is detected. Then an Approach Obstacle 
behavior follows another path to the region where the regularized sliding behav- 
ior becomes active. When the target can be reached by an optimal path, not 
intersecting the safety region around the obstacle (called Detect Target in the 
right figure). Approach Target becomes active again. In the right figure, the 
corresponding automaton is depicted. 



Thus, if we minimize f"{xdY instead of we make K{xd)^ small auto- 

matically, which is a desired feature, as seen in the previous paragraph. 

Since we, by following this proposed route, minimize the L^-norm of the sec- 
ond derivative, the resulting curve will be a cubic spline. This is a fortunate 
fact since it means that we will not be forced to relay on extensive world in- 
formation or to do any heavy computations on-line which tends to be the case 
when more sophisticated planning algorithms are used [12], [13], [18]. It is thus an 
almost trivial task to generate the splines that connect the robot and the target 
in the approach target behavior, and the robot and the obstacle in the approach 
obstacle behavior, as seen in Figure 8. 

3.2 Tracking 

We now have an on-line method for producing low curvature paths around de- 
tected obstacles, and hence our next task is to find a good tracking algorithm 
so that the robot follows the proposed path robustly. 

We let the general reference path, parameterized by s, be given by 




( 7 ) 



where the idea is to let the motion of the reference point be governed by a differ- 
ential equation containing error feedback. It can be viewed as a combination of 
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the conventional trajectory tracking, where the reference trajectory is parame- 
terized in time, and a dynamic path following approach [17], where the criterion 
is to stay close to the geometric path, but not necessarily close to an a priori 
specified point at a given time. This approach makes our algorithm robust to 
measurement errors and external disturbances since, if both the tracking errors 
and disturbances are within certain bounds, the reference point moves along the 
reference trajectory while the robot follows it within a prespecified look-ahead 
distance. Otherwise, the reference point should slow down and “wait” for the 
robot. 

Our control objectives are 

limsupi^^p(t) < Cp . . 

limsupi^^ |^(t) - (j)d{t)\ < C0, '' '' 



where Cp and are positive numbers that can be made arbitrarily small, p{t) = 
yj{xd — xY + {Vd — vY, where {x^y) is the actual position of the robot, and 4> 
and (j)d are actual and desired robot orientations. 

From (7) we directly get that id = p'{s)s,yd = q'{s)s, which implies that if 
the robot would track the path perfectly, we would have 



p'(s) 






F {s)+q' (s) p'^{s) + 



(9) 



since this corresponds to x = id and y = ijd- On the other hand, (9) does 
not contain any position error feedback, which is important for the robustness. 
Therefore we propose our dynamics for the reference point as follows: 



s = 




(10) 



where vq is the desired speed at which one wants the vehicle to track the path, 
and a and c are appropriate, positive numbers. 

We now let our control algorithm be as follows: 

u = 7pcos(e0) 

LV = ke^ + 4>d, fc > 0, 



where both 7 and k are positive, = 4>d — 4>, and (j)d = axcta,n2{yd — y,Xd~ x). 

In [4], [5] it was shown that for the platform model (3), governed by the control 
( 11 ), the steady state tracking error, p, can be made as small as one wants while 
(j) tends to (j)d exponentially. Furthermore, in steady state we have that v ^ vq. 
Thus we, by using the control law (11), meet the control objectives defined in 
( 8 ). 

We thus have a way of both producing and tracking paths, and we now 
combine these two together into the path following behavior that moves the 
robot safely around the obstacles at the same time as its executed trajectories 
are not too far from optimal with respect to curvature. As seen in Figure 9, 
where real experimental data are displayed, the method seems to work well. 
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(a) (b) 

Fig. 9. The results from implementing our ideas on the Nomad 200 can be seen. 
The reason why the sonar readings seem rather inaccurate is due to the fact 
that the robot has some drift in the odometry at the same time as the sonar 
resolution is rather coarse. 



4 Implementation 

There is a need to be able to define the hybrid automata in a structured and 
systematic way, making it easy to reuse and reorder nodes in different configu- 
rations. Therefore, at the Centre for Autonomous Systems (CAS) [2] at KTH, 
a programming environment for doing mobile manipulation^ within the hybrid 
automata framework has been developed [16]. It is called the MMCA, the Mobile 
Manipulation Control Architecture, and the core of the MMCA is an engine that 
executes hybrid automata, where, as mentioned, the nodes corresponds to dif- 
ferent behaviors. The architecture is designed to be open and allows the user to 
experiment with the contents of the behaviors freely, e.g. internal representations 
and algorithms, as long as the behaviors contain: 

(i) A function returning the desired state (in our case joint angles and plat- 
form pose) 

(ii) Conditions for when to make the discrete transitions 

A program written in the MMCA language begins with a specification of the 
initial node. Then all the nodes in the automaton are listed, where each node 
is specified by name, type, parameters and transitions. The transitions refer to 
the other nodes, or to itself, and the type of a node determines its functionality, 
such as what type of controller it is using, and it also defines which parameters 
or initial values that can be passed to the node. 

A sample file that defines the task of opening a door might look like 

INTERFACE = Puma560_XR4000 ; 

INITNODE = Approach; 

BEGIN 

NAME = Approach; 

^ From our point of view, this simply means that the mobile, behavior based platform 
has been angmented by the addition of a robotic arm, mounted on top [9] . 
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TYPE = Visual_Servo ; 



Object = Door_Hcindle ; “/« Servo on a door handle 

TRANSITION[End_Position] = Grasp; 

END 

BEGIN 

NAME = Grasp; 

TYPE = Grasp_Object ; 

Object = Door_Handle; “/« Grasp a door handle 
TRANSITION [Got_Grip] = Pull; 

TRANSITION [Lost_Gr ip] = Approach; 

END 

BEGIN 

NAME = Pull; 

TYPE = Follow_Arc; 



Radius = 0.8; % Estimate of the arc radius 

Angle = 90; 7o Open door 90 deg. 

TRANSITION [Ready] = End; “/« Terminates the control cycle 



END 



5 Conclusions 

In this article, it is shown that a behavior based control system can be modeled as 
a hybrid automaton, where each node corresponds to a distinct robot behavior. In 
order to achieve this, we have to impose hard switches on the transitions between 
the different behaviors, resulting in a potentially chattering overall behavior. 
We furthermore show how regularization techniques can be used to solve this 
problem by adding extra nodes to the automaton. Those extra nodes correspond 
to the sliding dynamics on the boundary between the different behaviors. The 
performance aspect of this approach is verified experimentally on a Nomad 200 
mobile platform. 

We also propose a heuristic method for designing reach target behaviors in 
such a way that questions concerning safety and optimality can be addressed 
explicitly. Our proposed method is based on a combination of path planning 
and trajectory tracking techniques, placing it in the deliberative part of the 
behavior based control architecture spectrum. Furthermore, we show that this 
approach works well in practice on our experimental platform. 

We conclude the article with a brief presentation of a programming environ- 
ment, the MMCA, for defining hybrid automata in a systematic and structured 
way. 
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Abstract. We consider hybrid systems consisting of a lower- level com- 
ponent with time-driven dynamics interacting with a higher-level compo- 
nent with event-driven dynamics. These typically arise in manufacturing 
environments where the lower-level component represents physical pro- 
cesses and the higher-level component represents events related to these 
physical processes. We formulate an optimization problem which aims 
at jointly optimizing the performance of both hierarchical components 
and present a hybrid controller for accomplishing this task. A numerical 
example is given to illustrate the operation of the hybrid controller. 



1 Introduction 

The term “hybrid” is used to characterize systems that combine time-driven 
and event-driven dynamics. The former are represented by differential (or dif- 
ference) equations, while the latter may be described through various frame- 
works used for Discrete Event Systems (DES), such as timed automata, max-plus 
equations, or Petri nets (see [5]). Broadly speaking, two categories of modeling 
frameworks have been proposed to study hybrid systems: Those that extend 
event-driven models to include time-driven dynamics; and those that extend the 
traditional time-driven models to include event-driven dynamics; for an overview, 
see [1][2][3][11]. 

The hybrid system modeling framework we will consider in this paper is 
largely motivated by the structure of many manufacturing systems. In these 
systems, discrete entities (referred to as jobs) move through a network of work- 
centers which process the jobs so as to change their physical characteristics 
according to certain specifications. Associated with each job is a temporal state 
and a physical state. The temporal state of a job evolves according to event- 
driven dynamics and includes information such as the waiting time or departure 
time of the job at the various workcenters. The physical state evolves according 
to time-driven dynamics modeled through differential (or difference) equations 
which, depending on the particular problem being studied, describe changes in 
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such quantities as the temperature, size, weight, chemical composition, or some 
other measure of the “quality” of the job. The interaction of time-driven with 
event-driven dynamics leads to a natural trade-off between temporal require- 
ments on job completion times and physical requirements on the quality of the 
completed jobs. For example, while the physical state of a job can be made ar- 
bitrarily close to a desired “quality target” , this usually comes at the expense of 
long processing times resulting in excessive inventory costs or violation of con- 
straints on job completion deadlines. Our objective, therefore, is to formulate 
and solve optimal control problems associated with such trade-offs. 

In this paper, we formulate and analyze a large class of optimal control prob- 
lems for hybrid systems viewed as consisting of two hierarchical components. The 
lower-level component represents physical processes characterized by time-driven 
dynamics, and the higher-level component represents events related to these 
physical processes. In the manufacturing context, jobs undergo various physical 
processes taking place at workcenters which are supervised through events such 
as starting and stopping the processes at appropriate times. Unlike earlier work 
in [9], which assumes a constant control input for each job and focuses on the 
optimization of the higher level component, we design a hybrid controller which 
has the task of communicating with both components and jointly solving cou- 
pled optimization problems, one for each component, hence outperforming the 
previous methods. To accomplish this objective, we will utilize techniques from 
classical optimal control theory (see [4] [12]) for the lower-level, along with re- 
cently developed optimization techniques (see [9]) for the higher-level viewed as a 
DES. A key difficulty we face for the latter is the presence non-differentiabilities 
in the event-driven state dynamics which limit the use of classical gradient-based 
techniques. Recently, however, it has been shown that approximating the event- 
driven dynamics using surrogate functions this difficulty can be overcome (see 
[ 6 ]). 

2 Problem Formulation 

The general hybrid system model we consider is illustrated in Fig. 1. A system is 
initially at some physical state Ci at time xq and subsequently evolves according 
to the time-driven dynamics 

zi= gi{zi,ui,t), zi(a;o) = Ci (1) 

where u\ is a control. In general, we write Ui(t) to allow for explicit dependence 
on time, but omit it here for notational simplicity. At time x\, a switch (event) 
takes place causing the physical state to become Z 2 {x\) = ^ 2 - In general, we 
allow for Z 2 {x\) yf zi(xi), and the physical state subsequently evolves according 
to new time-driven dynamics with this initial condition. The time of this switch, 
which we refer to as the temporal state of the system, depends on event-driven 
dynamics of the form 



xi = fo{xo,Zi,Ui,t) 



( 2 ) 




Hybrid Controllers for Hierarchically Decomposed Systems 119 



Physical State, z 




Fig. 1. Hybrid System Framework 



In general, after the ith switch, the time-driven dynamics characterizing the 
physical state Zi are given by 

— Hi t) , Zi{Xi^ — C^i (3) 

and the event-driven dynamics characterizing the switching times (temporal 
states) Xi are given by 

Xi = fi{xi-i,Zi,Ui,t) (4) 

Note that the choice of control following the zth switch affects both the physical 
state Zi and the next temporal state Xi+\ . Thus, the switches at times X\,X 2 , ■ ■ ■ 
are generally not exogenous events that dictate changes in the state dynamics, 
but rather temporal states intrically connected to the control of the system. We 
emphasize this fact since it is one of the crucial elements of a “hybrid” system. 
In some applications, the event-driven dynamics (4) may be viewed as exogenous 
switching times, substantially simplifying the analysis; this is not the case in the 
problems we tackle in what follows. 

In the context of manufacturing systems, the switches in Fig. I correspond 
to jobs that we index by z = I, . . . , iV. We shall limit ourselves to a single- 
stage process modeled as a single-server queueing system. The objective is to 
process N total jobs. The server processes one job at a time on a first-come first- 
served non-preemptive basis (i.e., once a job begins service, the server cannot be 
interrupted, and will continue to work on it until the operation is completed). 
Jobs arriving when the server is busy wait in a queue whose capacity is larger 
than N. 

As job z is being processed, its physical state, denoted by Zi € K (chosen 
scalar for simplicity), will be assumed to evolve according to LTI time-driven 
dynamics 

Zi = 9i{zi, Ui, t) = aZi + but, Zi{n) = Q (5) 

where Ti is the time processing begins and Q is the initial state at that time. The 
control variable Ui (assumed here to be scalar for simplicity) is used to attain 
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a final desired physical state corresponding to a target “quality level” . On the 
other hand, the temporal state of the ith job is denoted by Xi and represents the 
time when the job completes processing and departs from the system. Letting 
ai be the arrival time of the fth job, the event-driven dynamics describing the 
evolution of the temporal state are given by the following “max-plus” recursive 
equation: 

Xi = f{xi-i,Ui,t) = max(a;i_i,aj) -I- Si{u^) (6) 

where we set xq = — oo in which case x\ = a\ + si(ui) and the first job be- 
gins service as soon as it arrives. It is assumed that the job arrival sequence 
{ai,... jOat} and the initial conditions Ci for i = are given. The case 

where the order of jobs is not given is an alternative problem which we do not 
address in this paper. The recursive relationship (6) is known in queueing theory 
as the Lindley equation [5] and is the specific form of the event-driven dynamics 
(4) applicable to this particular hybrid system. 

This system is hybrid in the sense that it combines the time-driven dynam- 
ics (5) with the event-driven dynamics (6), the two being coupled through the 
choice of the control sequence {ui, . . . , mat} where Ui{t) is defined over an inter- 
val [max(a;i_i, Oi), Xj) which depends on the choice of Ui(t). The deterministic 
optimal control problem we consider has the general form 

N 

min J = Li{xi,Ui) (7) 

Ui,... .UN ' ^ 

subject to (5) and (6), where Li{xi,Ui) is a cost function associated with job i. 

We will concentrate on a family of problems for which the cost functions 
Li{xi,Ui) are separable in the sense that 

7^i{Xi^ V^z(^z) (8) 

The term ipi{xi) is the cost related to the zth job departing at time Xi. This cost 
may be associated with inventory level or tardiness of the job with respect to 
a required “due date.” For example, V'z(a^z) = (xi — XidY defines a cost where 
departing after the due date Xid incurs a tardiness cost and completing the job 
before due date incurs an inventory (backlog) cost. The term (j)i{ui,Si) includes 
the cost due to applying control Ui for Si units of time required to bring the 
physical state of the job as close as possible to a targeted “quality level” repre- 
sented by a desired final state. Unlike earlier work (e.g. [9], [8], [13]), we do not 
constrain the final physical state. Instead, the deviation of the departing job’s 
physical state from the desired “quality level” incurs a cost which is included in 
4>i{ui, Si). Thus, the optimization problem of interest is 

N 

min s^) + tpi{x^)] (9) 

Ui.... .UN ' ^ 

2=1 



subject to (5) and (6). 

In earlier work (e.g. [9], [8], [13]) the final state was fixed, therefore for given 
Mi, the processing time Si was uniquely determined. This simplified the analysis 
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because the cost on control could be written in terms of Ui only. However, in 
this paper, since the final physical state is not constrained, the service time Si 
is not uniquely determined by Ui. In what follows, we consider cost functions of 
the form 

1 1 

= -h{zfi- Zdt)'^ + J -ruj{t)dt (10) 

where a quadratic cost is imposed on the deviation of the final state from the 
desired value and on the control applied over a processing interval [0, Si). An 
additional quadratic cost on the physical state Zi{t) for t G [0,Si) may also be 
included; in manufacturing applications, however, it is typical that only the final 
state Zfi is of interest. 

In this setting, (9) is not easy to solve. Our approach is to uniquely determine 
Ui given Si by decomposing the hybrid system as explained next. 

Let us decompose the hybrid system hierarchically into two levels: At the 
lower level reside the time-driven dynamics based on which we need to control 
the physical state of each job to attain a target “quality level.” At the higher 
level reside the event-driven dynamics based on which service times are controlled 
over all N jobs. This decomposition is convenient because the optimization at 
the lower level can be done one job at a time (or in parallel over all N jobs), 
whereas at the higher level the optimization involves the coordination over all 
N jobs simultaneously. 

Lower-level problem. At the lower level, we consider a quadratic cost 

1 f®* 1 

= -^Hzf^ - Zdi)'^ + J -ruf(t)dt (11) 

which we view as a function of Sj, the time horizon available, i.e., the service 
time to be allocated to the ith job. Note that Zdi is a desired final physical state, 
Zfi is the actual final state, and h, r are weights associated with the terminal 
cost and control cost respectively. We choose the notation 6i{si) to differentiate 
this cost function from 4>i{ui, Si) in (10), since we will now seek to optimize over 
a given Si. In particular, we face an optimization problem for each f = 1, . . . ,N: 



min 9i{si) 

Ui 

s.t. Zi = aZi + but, Zi(0) = Ci (12) 

This problem can be solved as a function of Si, so that the optimal control is 
parameterized by Sj. Once the solution is obtained, we can evaluate the cost as 
a function of Si to get 

6*{s,) = min ^h{zf, - Zdif + ^ruf{t)dt (13) 

Higher-level problem. If the higher level is provided with the information 
u*(si), then 4>i{ui, Si) in (9) becomes 4>i{u*{si), Si), a function of the service time 
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Si- Let us denote this function by <j)i{si) and we are then faced with the following 
problem: 

N 

min 

si,....,Siv ^ ^ L J 

s.t. Xi = max{xi-i,ai) + Si (14) 

When this problem is solved, then the optimal values s* , . . . , s'^ can be commu- 
nicated to the lower level. Therefore, the optimal control values become u*{s*) 
for alH = 1, . . . ,N. Note that ^i(si) = 0*{si) for alH = 1, . . . ,N. 



3 Hybrid Controller 

The hybrid controller we propose for coordinating the two problems (12) and 
(14) outputs to the lower level the optimal controls u*{s*) for alH = 1, . . . , TV, 
and to the higher level the optimal service times s* for all z = 1, . . . ,iV. The 
operation of the controller is overviewed next in terms of its four basic steps. 
Note that, for simplicity, we assume that the desired final physical state for each 
job is Zd and the initial physical state is zq. 



Step 1: System Identification. The values of a and b in the physical state 
equation z = az + bu, the cost associated with the physical process <j), the desired 
final physical state Zd, and the initial state zq are input to the controller from 
the lower level. Similarly, the arrival sequence {ai), i = . . . , N, and the cost 

associated with the temporal states ip are also input to the controller. 



Step 2: Lower level controller evaluates 9*{si) and u*{si,Zd) for all 
z = 1, . . . ,N. The problem (12) is solved and the values of 0*{si) and u*{si, Zd) 
depend on the specific constraints imposed on the controls. If, for example, the 
controller can output arbitrary values at any time, the optimal control for this 
process can be obtained as the (transient) solution of a standard LQ problem 
(details are omitted) to give 



u* (t) = 2abe “*/z 



Zd - zoe“ 



2rae~“^* -I- 



(15) 



and the optimal final state is 

^ 2zora + e°-‘^^b‘^hzd — b‘^he~°'^'Zd 
2rae~“®* -I- e^^^b'^h — 



( 16 ) 



Therefore, 






u(t) ^2 * 



^di) 



-ruf(t)dt] 



= h 



ra{zd- 

2ra + — b'^h 
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By way of comparison, if the control is constrained to be constant (which is 
sometimes the case in manufacturing applications), its optimal value can be 
obtained as 



= - 1 ) 



Zd - zoe“ 



rsiO^ + /i62(e“Si — 1)2 



(17) 



which yields 



and 



hi I 

z}, = + -(e“^^ - 1) 

a 

rsja^ I -| 



\ izd-zoe‘^^*y 

Oi (si) = -h- 



2 . /Ib2(ea.,-i)2 ^ 

^ rsiO^ ' ’ 



(18) 



Other types of controllers, such as P, PI, and PID, are also applicable. They 
will have corresponding u*{si,Zd) and 0*{si) values depending on the solution 
of (12) using feasible controller outputs. 



Step 3: Higher level controller evaluates s* for alH = 1, . . . ,7V. Once 
the cost of service (j>i{si), which is equal to ^^(si), is known for some i, one can 
solve (14) and get the optimal service times s*. The solution to this problem is 
the topic of ongoing research with some results reported in [5], [8], [7]. Although 
the problem appears similar to classical discrete-time optimal control problems 
commonly found in the literature (e.g., [4]), there are two issues to address. First, 
the index i = 1, . . . , N does not count time steps, but rather asynchronously de- 
parting jobs. Second, the presence of the “max” function in the state equation (6) 
prevents us from using standard gradient-based techniques, since it introduces 
a non-differentiability at the point where = Xi-i. Regarding the first issue, 
although the absence of a synchronizing clock presents a difficulty encountered 
in all DES, note that the mathematical treatment of the recursive equation (6) 
is in fact no different than that of any other similar recursion where the index 
represents synchronized time steps as in classical discrete-time optimal control 
problems. Therefore, this issue is not really problematic. Regarding the second 
issue, recent work in [8], [7], [6] has led to the development of efficient algorithms 
that make use of non-smooth optimization techniques and exploit the structure 
of the problem. Alternatively, as described in [6], it is possible to approximate 
the corner of the max function with a differentiable surrogate, leading to very ef- 
ficient numerical solutions with little loss of accuracy. In particular, we “smooth” 
the max function by fitting it with a Bezier function at the neighborhood of the 
corner (see Fig. 2) and solve the resulting ‘Two Point Boundary Value Problem’ 
(TPBVP). 
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f{d , a) = max (d, a ) 

A 




a-e 



a a+e 



Fig. 2. Bezier approximation of a max function. 



A Bezier function is constructed using n + 1 “control points” represented by 
vectors Vi , . . . , Vji and is parametrically given by 

n 

v(i) = 

where 

72 1 . 

B^t) = - tr~^ 

i\[n — i)\ 

The control points define a “characteristic polygon” and the Bezier function has 
the property that it is contained within the convex hull of this characteristic 
polygon. In our case, there are three obvious control points to use: the point 
(a, a) where the max function is not differentiable and two points (a — e, a) and 
(a + e, a + e) which define a neighborhood of a on the d-axis in Fig. 2. The Bezier 
function in the neighborhood can therefore be formulated as 

v(t) = (a + e, a + e)t^ + 2(a, a)t(l — t) + (a — e, a)(l — 

= (a + — 1), a + et^) 

where t G [0, 1]. The derivative of the Bezier function is 

d(a + et^)/dt 
d{a + e(2t — l))/dt 

i.e., it starts at v(0) =(a — e,a) with derivative 0 and ends at v(l) =(a + e,a + e) 
with derivative 1, coinciding with the derivatives of the max function at these 
control points. Note that the derivative of the Bezier approximation of the max 
function stays between 0 and 1 inside the characteristic polygon. 
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Fig. 3. Hybrid Controller Operation 



The value e > 0 determines how tightly the surrogate function fits and can 
be adjusted during the execution of the TPBVP solver to achieve any desired 
accuracy. Selecting a very small e at the beginning of the algorithm may result in 
chattering; therefore, it is more desirable to gradually decrease e as the algorithm 
approaches the optimal. In the limit, the solution obtained using this approach 
converges to the true optimal. 

Step 4: Optimal controls are output to lower level. Once the optimal 
service times s*, i = 1, . . . , N, are determined, they are provided to the lower 
level controller that obtained u*{si, Zd) at Step 2. The final values of the optimal 
controls are u*{s*,Zd), i = 1, . . . ,N, and these are issued to the lower level for 
controlling the physical processes. 

During normal system operation, the controller supplies the optimal control 
sequence {u*{s*,Zd)} to the physical (lower) level and the optimal service time 
sequence {s*} to the higher level (see Fig. 3). Based on {s*} and the arrival time 
sequence {ui}, the higher level can signal the lower level when to start and when 
to stop the ith process. 

4 Numerical Example 

In order the illustrate the operation of the hybrid controller, we consider a single- 
stage hybrid manufacturing system which incurs the cost 

N 

J(u,x) = -\- ^pi{x^) 

s.t. Xi = max(a;i_i, -I- Si 



while processing N jobs. 
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In this example, we will assume that, for z = 1, ..., N, the cost '4’ii.^i) associ- 
ated with the temporal state Xi is 

V'i(xj) = !3{xi - aif- 



which penalizes the system time of the zth job. The cost of processing the zth 
job, (j)i{ui, Si), is in the form of (10). If, in addition, the physical state of the zth 
job evolves according to 

Zi = Ui , Zi{xi) = q 



where we assume the final state to be fixed, then the lower-level cost 9i{si) in 
(11) becomes 








For simplicity, we assume fixed initial state Cz = 0 for z = 1, The arrival 
sequence {oi} is also given. 

Step 1: System Identification The values a(= 0),6(= 1), the cost func- 
tions 9{si) and tp{xi){= j3{xi — a^)^) and the sequence {ai} are passed to the 
hybrid controller. 

Step 2: Lower level controller evaluates 9*{si) and u*{si,Zd) for all 
z = 1, . . . , N. The Hamiltonian for the lower-level component is defined as 

H{t) = ^ru^,{t) +p{t)ui{t) 



where p{t) is the co-state, hence the necessary conditions for optimality are 

n 



Therefore, 

(^) = — - — u* (constant) 



Integrating the state equation Zi = Ui gives 




The optimal control, therefore, will incur a cost 



Hsi) = 9*{si) 




ru^dt 



q 

— Si= JUi 
Si 



where 7 = ^rq. 
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Step 3: Higher level controller evaluates s* for all z = 1, . . . , iV. The 

higher level optimization problem becomes 



N 



J(u, x) = min^ 7 Ui + f3{xi — ai)^ 



subject to 



Xt = maxtxi-i, Oj) H 

Ui 



Let us form the augmented cost 

N 

J(u, X, A) = miny^ -fUi + (3{xi — aif + Aj(max(xj_i, a,) + — — Xi) 

7/ • 



7=1 



The optimality equations are 



which yield 



BT BJ 

— = 0, ^ = 0, forz=l,...,7V 

oui Bxi 



dJ _ X 7 _ n ^ 2 _ 

— 7 2 — — 

oui uf 7 



Az — \i 



i+1- 



max(a;i, Oi+i) 



dxi 

\n = ‘2P{xn — o:n) 



2(3{xi — ai) for i < N 



Using the Bezier approximation approach described in the previous section, this 
TPBVP can be solved effectively to evaluate the optimal service time sequence 
{<}• 



Step 4: Optimal controls are output to lower level. The optimal con- 
trol input M* = — is fed to the system while processing the zth job (during 
[max(xi_i, Oi), a;i) interval) which departs at time Xi. 

Example 1. Consider the one-stage system where fV = 10 jobs all arrive at time 
t = 0. Ifr = 6, q = 10, P = 1 then the optimal controls and service times are as 
follows. 

Job Service Time Optimal Control Departure Time 



1 


1.35 


7.43 


1.35 


2 


1.36 


7.37 


2.70 


3 


1.38 


7.25 


4.08 


4 


1.42 


7.06 


5.50 


5 


1.47 


6.79 


6.97 


6 


1.55 


6.44 


8.52 


7 


1.67 


5.98 


10.20 


8 


1.86 


5.38 


12.06 


9 


2.18 


4.58 


14.24 


10 


2.95 


3.39 


17.19 
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This control results in a cost of J* = 2774.83. 

Note: The time requirement of our algorithm on a standard PC is of the 
order of seconds. Readers are referred to the web site 

http: / /vita. bu. edu/ cgc/ newhybrid/ onestage.html 

to reproduce this example or try different examples by interactively varying the 
arrival sequence and other problem parameters. 

5 Conclusions 

In this paper, we considered hybrid systems modeled as a two-level hierarchy 
and hybrid controllers that were designed to jointly optimize the performance of 
both levels. The lower-level optimization problem, i.e., the determination of the 
optimal control u*(t) for the physical process of each job i when the service time 
Si is known, employs classical control techniques. The higher-level optimization 
problem, i.e., the determination of the optimal service time sequence {s^} when 
the cost of each service time //s/) is known, employs recently developed opti- 
mization techniques for DES (see [9]). The result of one optimization problem is 
the input to the other, therefore these optimization problems are highly coupled. 
The key to the decoupling process is the following: Since the lower-level controller 
knows the form of the optimal control solution for the deterministic process, it 
passes the cost information //s/) to the higher-level controller. The higher-level 
controller can then determine the optimal service sequence {sj} which is passed 
to the lower-level controller for determination of optimal controls u*(t) to the 
physical processes. 

This decomposition method relies highly on the deterministic structure of the 
physical processes. In the case where the arrival sequence is not known, one can 
start with the mean value information and resolve the optimization problem as 
the arrivals are observed. The speed of the solution algorithm in such a method 
is a key issue and is the subject of ongoing research. Another interesting case 
where the physical processes are stochastic was considered in [10] and is also a 
topic of ongoing research. 

The idea of decomposition is not limited to the specific class of problems 
presented in this paper. The event driven dynamics at the higher-level and the 
time driven dynamics at the lower-level can be arbitrary. 
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Abstract. Since hybrid embedded systems are pervasive and often safety- 
critical, guarantees about their correct performance are desirable. The 
hybrid systems model checker HyTech provides such guarantees and has 
successfully verified some systems. However, HyTech severely restricts 
the continuous dynamics of the system being analyzed and, therefore, 
often forces the use of prohibitively expensive discrete and polyhedral 
abstractions. We have designed a new algorithm, which is capable of di- 
rectly verifying hybrid systems with general continuous dynamics, such 
as linear and nonlinear differential equations. The new algorithm con- 
servatively overapproximates the reachable states of a hybrid automa- 
ton by using interval numerical methods. Interval numerical methods 
return sets of points that enclose the true result of numerical computa- 
tion and, thus, avoid distortions due to the accumulation of round-off 
errors. We have implemented the new algorithm in a successor tool to 
HyTech called HyperTech. We consider three examples: a thermostat 
with delay, a two-tank water system, and an air-traffic collision avoid- 
ance protocol. HyperTech enables the direct, fully automatic analysis 
of these systems, which is also more accurate than the use of polyhedral 
abstractions. 



1 Introduction 

In a hybrid system, digital controllers interact with a continuous environment. 
Because of the increasing ubiquity of embedded real-time systems, hybrid sys- 
tems directly control many of the devices in our daily lives. Moreover, hybrid 
systems are often components of safety- or mission-critical systems. For these 
reasons, it is necessary to have rigorous guarantees about the correct perfor- 
mance of hybrid systems. 

* This research was supported in part by the DARPA (NASA) grant NAG2-1214, the 
DARPA (Wright-Patterson AFB) grant F33615-C-98-3614, the ARO MURI grant 
DAAH-04-96-1-0341, and the NSF CAREER award CCR-9501708. 



N. Lynch and B. Krogh (Eds.): HSCC 2000, LNCS 1790, pp. 130—144, 2000. 
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Hybrid automata [1] provide a modeling paradigm for hybrid systems. In a 
hybrid automaton, the discrete state and dynamics are modeled by the vertices 
(called locations) and edges of a graph, respectively, and the continuous state and 
dynamics are modeled by points in M” and differential equations, respectively. 
Symbolic model checking on a hybrid automaton provides correctness guaran- 
tees. HyTech [10] is a model checker for hybrid systems that has been successful 
in analyzing many hybrid systems of practical interest [2,5,13,14,15,16,23,26,27]. 

Despite its successes, HyTech has several shortcomings. It restricts the dy- 
namical model of the automaton being analyzed to that of linear hybrid au- 
tomata. In linear hybrid automata, the continuous dynamics are governed by 
polyhedral differential inclusions, and all trajectories are composed of lines with 
piecewise constant slopes. These limitations force the verifier to approximate 
the complex dynamics of a hybrid system in a less expressive dynamical model. 
This approximation may take the form of rate translation [11], in which the first 
derivative of every continuous variable is bounded above and below by constants. 
Location splitting may be used to make the approximation arbitrarily accurate: 
each location can be split into many new locations; in these new locations, the 
dynamics may be bounded more precisely. However, location splitting leads to 
state explosion, as accuracy in the model comes at the price of a large number 
of new locations. Thus, the restrictive input language often forces the use of 
prohibitively large approximate models. 

A second deficiency of HyTech is that arithmetic overflows frequently occur 
in the course of HyTech’s computation. To explain this problem, we briefly de- 
scribe the basic algorithm underlying HyTech. A state s of a hybrid automaton 
has two types of successors: flow successors, which are the states reachable from 
s by letting time progress; and jump successors, which are the states reachable 
from s if the automaton undergoes a change of location. Call a set of states a poly- 
hedral region if its continuous part is a polyhedron. For linear hybrid automata, 
the flow and jump successors of a polyhedral region form again polyhedral re- 
gions. The computation engine of HyTech computes the set of states that can be 
reached from an initial polyhedral region by any number of flows and jumps. The 
iterated computation of flow and jump successors continues until either a target 
state is reached or no new states are generated.^ The polyhedral manipulations 
for computing flow and jump successors use exact computation over rationals 
stored as integer pairs. However, these repeated computations quickly generate 
rationals with very large representations, leading to arithmetic overflows. 

We have implemented the program HyperTech, which addresses both in- 
adequacies of HyTech. First, HyperTech supports the analysis of hybrid au- 
tomata with much more general dynamics. In particular, HyperTech can ana- 
lyze automata whose continuous dynamics are given by differential equations of 
the form dxi/dt = f{xi , . . . , x„), where / is a composition of polynomials, expo- 
nentials, and trigonometric functions. This class of hybrid systems includes all 
multi-modal linear systems, i.e., systems whose continuous dynamics are given 

^ In general, the computation may fail to terminate, because the reachability problem 
for linear hybrid automata is undecidable [1]. 
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by matrix differential equations of the form dx/dt = Ax + Bu (where u rep- 
resents the control input or disturbance input). HyperTech’s more permissive 
input language enables a direct modeling of the continuous dynamics of hybrid 
systems. Since the need for introducing abstractions (in the form of rate trans- 
lation and location splitting) is removed, input automaton models can be much 
more compact than with HyTech. 

Second, HyperTech uses interval numerical methods [20,22] to compute 
an overapproximation of the set of reachable states of a hybrid automaton. In 
interval methods, the computed solution to a numerical problem, e.g., an initial 
value problem, is guaranteed to enclose the true solution. This is in contrast to 
conventional numerical methods, in which the accumulation of round-off errors 
may cause a computed solution to deviate from the real solution. The analysis 
engine of HyperTech, like that of HyTech, starts with an initial region and 
iteratively adds ffow and jump successors. However, HyperTech uses an interval 
ordinary differential equation (ODE) solver, instead of polyhedral manipulations, 
to compute an overapproximation of the ffow successors of a set of states. It is the 
use of interval numerical methods which guarantees that the reachable states of a 
hybrid automaton H are contained in the set of states computed by HyperTech 
when run on H . All regions resulting from interval methods are rectangular, i.e., a 
product of intervals. Since geometrically manipulating rectangles is simpler than 
manipulating arbitrary polyhedra, the internal representations of HyperTech’s 
rectangles never grow very large. In this way, HyperTech avoids the numeric 
overflow errors of HyTech. 

In essence, while the restrictive dynamics of HyTech force an approximation 
in the model {static approximation), the permissive dynamics of HyperTech 
allow approximation to occur only during the computation of reachable states 
{dynamic approximation). Despite the fact that the dynamic approximation us- 
ing rectangular regions seems rough, we demonstrate it to be superior to static 
approximation using polyhedral differential inclusions, on three examples: a ther- 
mostat with delay, a two-tank water system, and an air-traffic collision avoidance 
protocol. 

In traditional numerical integration, the accumulation of round-off errors may 
cause the computed solution to a numerical problem to differ widely from the real 
solution. In the context of hybrid systems analysis, the loss of precision caused 
by the accumulation of round-off errors in the numerical integration process 
may lead the analyzer, whether human or computer, to overlook potentially 
hazardous events. In contrast to hybrid systems simulators (see [21] for a survey) 
and reach-set computation tools [3,4,6,8,26] which use traditional (not interval- 
based) numerical methods, HyperTech is guaranteed not to miss any events. 
Thus, if an unsafe (target) state of a hybrid automaton is reachable, HyperTech 
is guaranteed to note its reachability. 

The rest of the paper is organized as follows. In Section 2, we describe the 
syntax and semantics of the hybrid automaton model, and define the reachabil- 
ity problem. In Section 3, we describe in detail the algorithm implemented in 
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HyperTech. In Section 4, we describe the results of running HyperTech on 
three examples, and compare the results with HyTech. 



2 Hybrid Automata 

To model hybrid systems, we use hybrid automata [1]. Let M” be the n-dimensio- 
nal Euclidean space. A rectangle of dimension n is a subset of K" that is the 
Cartesian product of (possibly unbounded) intervals, all of whose finite endpoints 
are rational. For a positive integer n, let 7^" denote the set of all n-dimensional 
rectangles. An axis-parallel hyperplane h C R” is a set of points {x | Xj = a} for 
some i G {1, . . . ,n} and some rational number a. Let a dynamical equation be 
an expression A generated by the grammar 

A := (ii = Hi A ±2 = A • • • A = Bi ) , B := Xj \ [a, b] \ Bi op B 2 \ f(H) 

where i and j are positive integers, a and b are any rational numbers such that 
a < b, op is one of the arithmetic operations + (addition), — (subtraction), • 
(multiplication), / (division), or " (exponentiation), and f is one of the functions 
sin, cos, tan, or exp. We shall use conventional mathematical notation for dy- 
namical equations whenever possible, and if a = 6 we shall often omit the square 
braces. For example, xi = ^\/xi — A X 2 = i/xf -h X 2 is a dynamical equa- 
tion. For a positive integer n, let 5” denote the set of all dynamical equations 
in which for each subexpression of the form Xi or xj, both i < n and j < n. The 
above example of a dynamical equation is a member of . 



2.1 Syntax 

A hybrid automaton H consists of the following components: 

— A finite set X = {a;i, . . . ,x„} of real- valued variables. A valuation of these 
variables represents a continuous state of a hybrid system. 

~ A finite directed multigraph (V,E). The vertices in V (called control loca- 
tions) represent the discrete state of a hybrid system. The edges in E {control 
switches) represent transitions between discrete states. 

— Three functions inv : V 7^”, init : V — > 7^”, and flow : V — > 5”. 
Each invariant inv{v) represents a condition that must be satisfied if the 
automaton is to remain in location v. Each initial condition init{v) C inv{v) 
represents the continuous states in which the hybrid automaton may begin 
executing, when control starts at location v. Each flow condition flow{v) 
constrains the continuous dynamics of the hybrid system at location v. 

— Two functions pre : E TiA and post : E — > 7^”. For each edge e = {v,v') 
in E, we require that pre{e) C inv{v) and that post{e) C inv{v'). Intuitively, 
pre{e) represents the condition on the continuous state that must hold if 
control is to pass from v to v' , and post{e) constrains the possible values of 
the variables after the transfer of control from v to v' . 




134 



Th.A. Henzinger et al. 




Fig. 1. Thermostat with delay 



— A function update : E — > that assigns to each edge e = (u,u') G 

E a subset update{e) C After traversing e, if the index i is in 

update{e), then the variable Xi gets nondeterministically reset so as to lie 
in the i-th projection of post{e), whereas if z ^ update{e), then Xi remains 
unchanged. 

— A finite set E of events, and a function event that assigns to each edge e G E 
an event. 

As an example, consider the hybrid automaton of Figure 1, which models a 
thermostat system with delays: after the thermometer detects that the temper- 
ature is too low or too high, there may be a delay of up to one second before 
the appropriate control action (turn the heater on or off, respectively) is taken. 
The variable xi measures the temperature. Initially, xi = 2 and the heater is 
on. The temperature rises according to the differential equation xi = —xi + 4. 
Eventually, the temperature reaches three degrees; after a delay of one second in 
location delayi, the thermostat sends a turn. off signal to the heater. The vari- 
able X 2 measures the delay. The temperature then falls according to the equation 
Xi = —Xi until xi = 1. One second after the temperature reaches one degree, the 
thermostat sends a turn.on signal to the heater, and the run of the automaton 
continues. 



2.2 Semantics 

We now give a formal definition of the semantics of a hybrid automaton. A state 
of a hybrid automaton is a pair (u,x), with location v G V, continuous state 
X G K", and x satisfying inv{v). The state space of a hybrid automaton is the 
set of its states. If u G K” is a vector, we denote by AT := u the interpretation 
for the variables in X in which Xi = for z = 1, . . . , n. A hybrid automaton has 
two types of transitions: 
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~ Jump transitions, which correspond to instantaneous transitions between 
control locations. Formally, there is a jump transition from state {v, x) to 
state (u',x') if there is an edge e = {v,v') G E with x satisfying pre{e), and 
x' satisfying post{e), and x' = x^ for i ^ update{e). 

— Flow transitions, which correspond to the continuous evolution of the system 
at a single control location v according to the dynamics specified by flow{v). 
Formally, there is a flow transition of duration t > 0 from state (f,x) to 
state (n,x') if there is a differentiable function / : [0,t] ^ M" such that: 
(1) /(O) = X, f{t) = x'; (2) for all reals t' € [0,t], /(F) G inv{v); and (3) for 
all reals t' G [0,t], the interpretation X,X := /(F), /(F) satisfies flow{v). 

We say that (v',x') is a flow (respectively jump) successor of (n,x) if there is 
a flow (respectively jump) transition from (u,x) to (u',x'). A run of a hybrid 
automaton is an infinite sequence of states (uo,x°), (ui,x^), . . . such that x° G 
init{vo), and for alH > 0, (ui+i, x*+^) is a jump or flow successor of (ui,x*). 

2.3 Reachability Problem 

The fundamental verification problem for hybrid automata is safety verification: 
given a partition of the state space into “safe” states and “unsafe” states, verify 
that each execution of the hybrid automaton does not reach the unsafe states. 
Dually, one may look at the reachability question: does any run of the hybrid 
automaton ever reach an unsafe state? Formally, given a hybrid automaton H 
and a subset S of its state space, the reachability problem asks if there is a 
run (uo,x°), (ui,x^), ... of H such that (ui,x*) G S for some i. If there is such 
a run, we say that the set S is reachable. Clearly, a solution to the reachability 
problem gives a solution to the safety verification problem as well. The reacha- 
bility problem is undecidable even for simple subclasses of hybrid automata [12]. 
However, semidecision procedures — for example, the algorithm of HyTech — 
often terminate on specific problems of practical interest. 

3 The HYPERTECH Algorithm 

3.1 Interval Numerical Methods 

In numerical computations, such as the numerical solution of ODEs, rounding 
errors may distort the accuracy of a sequence of calculations. Thus, ordinary 
numerical methods cannot provide fully rigorous guarantees about the safety 
of dynamical systems. Interval numerical methods [20] address this problem by 
computing sets of points that contain the true solutions to a numerical problem. 
In particular, interval ODE solvers find guaranteed bounds for the solutions to 
initial value problems. 

In interval methods, the fundamental object of computation is not a floating 
point number, but rather an interval. An interval [x, ir] is a nonempty set of 
real numbers {xGM|£<a;<iE}, where x < x are both real numbers. One 
can extend to intervals the usual arithmetic operations over reals: if op is an 




136 Th.A. Henzinger et al. 



arithmetic operation, then [x,x] op [y^y] = {x op y \ x G [^,x],y G [y,y]\. 
The operations and / on intervals may be seen to satisfy the following 

identities: 

k, + [y , y] = [x + y,x + y\ 

k, - [y , y] = [x-y,x-'^ 

[s, -11,^] = [min(£ ■ y,x-y,x ■ y,x -y), max(x ■ y,x-y,x ■ y,x -y)] 

1/ [x,x] = [1/x, l/x] if 0 ^ [$., S’] 

A computer implementation of these operations sets the processor’s rounding 
mode to round down when computing the lower bound of the result, and round 
up when computing the upper bound. This guarantees that the computed result 
always encloses the result that would have been obtained using exact arithmetic 
calculation. In a similar fashion, one can implement interval versions of standard 
functions (e.g., sin a;, e^, etc.) so that the computed result contains the exact 
result. Several interval arithmetic packages exist, either as libraries [18] or as 
extensions to regular programming languages [17]. 

Interval methods to solve initial value problems use as primitives the interval 
operations +, — , •, and / defined above, plus interval implementations of stan- 
dard functions such as sine and cosine. From an initial condition (a rectangle 
To at time 0), these methods usually compute a rough enclosure XAt of the so- 
lution at time At, where At is an input parameter to the program. This rough 
enclosure, which is a rectangle, is usually narrowed by a pruning procedure that 
reduces the accumulation of numerical errors, and mitigates the wrapping effect. 
(The wrapping effect is the error resulting from enclosing a nonrectangular re- 
gion by a rectangle.) This iteration — computing XiAt using X(^i_i)At by finding, 
and then pruning, a rough enclosure at time iAt — continues for a number of 
steps which is specified by another input parameter. 

Several implementations of interval ODE solvers are publicly available, for 
example [19,24]. These typically use Picard iteration to prove the existence and 
uniqueness of a solution, and to find a rough enclosure. This enclosure is then 
pruned both by using a mean value method and by bounding the error term in 
a truncated Taylor expansion. To reduce the wrapping effect, local coordinate 
transforms may be applied. For a variety of examples, these implementations 
find fairly tight solution enclosures. In our implementation, we have used the 
ADIODES library [24]. Our choice of this library is independent of the other 
parts of HyperTech; thus, any other interval ODE solver, e.g., AWA [19], may 
be used in place of ADIODES. 

3.2 Overapproximating Reachable States 

For a complex hybrid automaton H , precise analytic or closed- form descriptions 
of the reachable states of H may not exist or may be extremely difficult to find. In 
such cases, one must seek feasibly computable approximations of the reachable 
states. An overapproximation of the reachable states of is a superset T of 
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the reachable states of H. For analysis of the safety of a hybrid automaton, 
such an approximation may be useful, since if no unsafe state is in T, then no 
unsafe state is reachable. However, since there may be states in T which are not 
reachable states of H, the presence of an unsafe state in T does not necessarily 
imply that an unsafe state of H is reachable. In such cases, one could try to 
refine the automaton under consideration. 

Alternatively, one could compute overapproximations of the states which 
are backward reachable from the intersection of T and the unsafe states. If no 
initial state is contained in this backward approximation, then no unsafe state 
is reachable. This process may be iterated to find closer approximations of the 
reachable unsafe states [7,9]. It is an interesting question in its own right to 
determine whether an error run produced by an overapproximative algorithm is 
an actual error run. 



3.3 Overapproximation Using Interval Numerical Methods 

For a hybrid automaton with discrete state set V and n real- valued variables, let 
a region be a set of states of the form {u} x U, where C/ is a rectangle in R”. For 
any control location v GV, let Hv be the set ^ pre{v,v') \ {v,v') G E}. 

For a rectangle U and a location v, let UAt,v be the points x' G R” such that 
there is a flow transition from (v, x) G 17 to (u, x') of duration At. The procedure 
of HyperTech, which is presented in Figure 2, works as follows. It maintains 
two sets of regions: Reached, the explored set of regions, and Frontier, the set 
of regions that still need to be explored. As long as Frontier yf 0, one member 
{u} X [/ of Frontier is selected and removed from Frontier. The rectangle U is 
propagated according to the dynamics of v. An overapproximation of the set of 
reachable states (v, x) is added to Reached, and an overapproximation of the set 
of reachable states (u',x) (with {v,v') G E) is added to Frontier. 

The subroutine Propagate ^ first computes Y, a rectangular overapproxima- 
tion of UAt,v HyperTech uses an interval ODE solver to compute this overap- 
proximation. The size of At must be determined by the user. Let S be the set 
of points reachable for some At < At, i.e., S = Uo</iT<zit Uat,v In addition to 
Y , the interval numerical method generates a rectangle T that contains the set 
S. In the procedure, New-Reached gets set to an overapproximation of the set of 
states in {u} x S. Moreover, New-Frontier gets set to an overapproximation of 
the jump successors of states in {u} x S. Notice that for large values of At, this 
bound on S may be quite coarse, and may not suffice to prove the safety prop- 
erty of interest. In that case, we have to reduce At and run the procedure again. 
Thus, whereas computations will be faster for larger values of At, more accurate 
analysis may require smaller values. (This speed/ accuracy tradeoff is illustrated 
in Figure 6.) We wish to emphasize that our procedure is sound regardless of 
which At > 0 is chosen. 

Theorem 1. Let H he a hybrid automaton, and let (v,x) he a reachable state of 
F[ . If the procedure Reachable-States (using subroutine Propagate-y) terminates 
on H, then (v,x) G IJ Reached. 
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Reachable- States {H : hybrid automaton) 

Initialization: Frontier := {{«} x init(v) \ v € V^} ; 

Reached := Frontier-, 
while Frontier 7^ 0 do 

pick ({«} X U) :€ Frontier-, 

Frontier -.= Frontier \ ({«} x U) ; 

{New-Reached, New- Frontier) := Propagate (v, U ) ; 
Reached := Reached U New- Reached-, 

Frontier := Frontier U New- Frontier-, 
end while; 



Propagate ^{v : location, U : rectangle) 

Y := a rectangular overapproximation of UAt,v-, 

T := a rectangle which contains inv{v) Pi (^Uo<AT<At ^AT,v'j ; 
New-Reached := {{«} x T} ; 

New-Frontier := 

{Unexplored-, Jump- Successors (v,T)) U ({u} x {inv{v) n Y )) ; 
return {New-Reached, New-Frontier)-, 



Unexplored- Jump- Successors {v : location, T : rectangle) 

return {{«'} x Z \ {v, v') € E, Z = Update{T n pre{v, v')), 
Z / 0, ({«'} X ^) g U Reached} ; 



Fig. 2. HyperTech’s procedure for reach-set computation 



While Propagate I performs only one time step computation, under additional 
assumptions it is possible to group together multiple time step computations. 
The resulting procedure, called Propagate 2 , is shown in Figure 3. In order for 
the subroutine Propagate 2 to function correctly, the hybrid automaton H must 
satisfy the following conditions: (1) for each edge {v,v') € E, the rectangle 
pre{v, v') is a boundary of the invariant inv{v); and (2) for each control location 
V and each point x S inv{v), there exists a unique edge e = {v,v') such that, 
under the dynamics flow{v), the point x moves strictly monotonically towards 
the hyperplane pre{v,v'), and x eventually crosses pre{v,v'). For a large class 
of examples, including the hybrid automata in this paper, these two conditions 
hold. 

Note that the above conditions imply that transitions are urgent — they must 
be taken as soon as they are enabled. Thus, Propagate 2 needs only to consider the 
first time a region hits one or more exit hyperplanes. The subroutine Propagate 2 
functions like multiple iterations of Propagate i, except that at each iteration 
those trajectories which have crossed an exit hyperplane are not further explored. 
By the conditions above, this optimization does not compromise soundness — the 
procedure of HyperTech still explores all reachable states. 
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Propagate 2 (v : location, U : rectangle) 

W := U-, New-Reached ~ New-Frontier := Wprev ~ T ~ P := 0; 
while lU 7 ^ 0 do 

Wprev ■■=W-, 

W := Si rectangular overapproximation of WAt,v', 

T := Si rectangle which contains inv{v) Pi (^Uo<AT<At ; 

New-Reached := New-Reached U {{i>} x T} ; 

P := the subset of W that has crossed Nv', 

W :=W\P-, 

if P yf 0 then New-Frontier := 

New-Frontier U Unexplored- Jump-Successors{v,T) endif; 

end while; 

return {New-Reached, New-Frontier)-, 



Fig. 3. Grouping together multiple time step computations 



Theorem 2. Let H he a hybrid automaton satisfying conditions (1) and (2) 
above, and let (u,x) be a reachable state of H. If the procedure Reachable -States 
(using subroutine Propagatc 2 ) terminates on H, then (u,x) G [J Reached. 

4 Three Examples 

With the use of interval methods, we obtain both a more direct model of the 
target system (i.e., no rate translation needed) and tighter bounds on the sets of 
reachable states. We substantiate this claim by describing the results of running 
HyperTech on three examples. 

4.1 Thermostat With Delay 

Consider again the hybrid automaton of Figure 1. We wish to determine the 
range within which the temperature always lies. The nonlinear dynamics can- 
not be modeled directly in HyTech. Instead, the dynamics of the temperature 
xi are approximated using rate translation [11]. Using this method, the bounds 
obtained by HyTech are 0 < xi < 4. This approximation may be made arbi- 
trarily accurate by splitting each control location and using better bounds on 
the derivatives in the new locations. By combining rate translation with loca- 
tion splitting, and using a 20-location approximation of the system, HyTech 
obtains the bounds 0.28 < xi < 3.76. This 20-location automaton is pictured in 
Figure 4. 

We can run our algorithm directly on the automaton of Figure 1, with a 
step size of At = 0.1. Initially, x\ = 2, and the automaton is in location on. 
Our algorithm propagates the values of xi according to the differential equation 
xi = —x\ -\- 4, until the interval containing the true value of x\ entirely crosses 
the exit condition x\ = 3. At this point, there is a discrete jump to location 
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Fig. 4. Rate translation of thermostat automaton, with each location split into 
five locations 




4 



Fig. 5. Two-tank system 



delayi. Now our algorithm propagates the interval [3,3] for one time unit. At 
the end of one time unit, X\ < 3.64, and the automaton jumps to location off. 
Continuing this process, our algorithm reports that the minimum value of xi 
(which is reached in location delay 2 ) is 0.367. Therefore, using HyperTech, the 
bounds are 0.367 < < 3.64. The bounds found by analytically solving this 

system are ^ < 4 — i. Note that ^ « 0.3679 and 4 — i « 3.632. Comparing 

our results with the analytic solution shows that HyperTech computes a close 
approximation to the actual set of reachable states. 



4.2 Two- Tank System 

As a second example, we consider the two-tank system of [25] (see Figure 5). The 
plant consists of two identical interconnected tanks. Into tank 1 flows a stream 
characterized by the loss parameter ki.^ Tank I’s outlet stream, characterized 
by the loss parameter k 2 , flows into tank 2. Tank I’s outlet stream is fcs meters 

This loss parameter may be thought of as a friction loss term. 



2 
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Fig. 6. A portion of the generated rectangles for the two-tank system at times 
iAt, for z = 0, 1, 2, ... . HyperTech’s actual computed overapproximation is the 
union of all rectangular hulls of pairs of consecutive rectangles. The horizontal 
(resp. vertical) axis shows the values of x\ (resp. X 2 )- From the left: At = 
5, running time: 24.27 s.; At = 2, running time: 53.39 s.; At = 1, running 
time: 98.60 s.; and At = 0.5, running time: 190.64 s. 



above tank 2. The outlet stream of tank 2 is characterized by loss parameter k^. 
Let x\ and X 2 denote the heights of the liquid columns in tank 1 and tank 2. 
Applying Toricelli’s law, the dynamics of this system may be seen to be: 

{ ( k i - k2\/xi -X 2 + ks \ 

[ k2\/xi -X2 + k3- k4y/^ ) ^ ^23 
/ ki- k2y/^ 

\k2y/¥^ - kAy/^ 

The dynamical equations change when the liquid level in tank 2 is equal to the 
height of the connecting pipe. Under this dynamics, the system moves towards an 
equilibrium point for all Xi > 0 and for all ki > 0. For example, for the parameter 
values k 2 = k 4 = 1 \J meters per second, kz = 0.5 meters, and k\ = 0.75 meters 
per second, the system moves towards the equilibrium point x\ = 0.625 . . . , X 2 = 
0.563 .... In [25], rate approximation is used to model this dynamical system 
as a 12-location hybrid automaton; HyTech is then used to overapproximate 
which states were reachable. With HyperTech, we directly model the system 
as a hybrid automaton with two states, corresponding to whether X 2 > fcs or 
not. Further, the analysis is more accurate. For example, HyTech’s analysis 
of the 12-location rate approximation finds that starting from 0.70 < x\ < 
0.80 and 0.45 < X 2 < 0.50, some states in which both 0.60 < a;i < 0.80 and 
0.60 < X 2 < 0.65 are reachable, whereas our algorithm shows that these states 
are unreachable. In Figure 6, we show a part of the overapproximation of the 
reachable states of the two-tank system, for four different choices of the time 
step At, with the corresponding running times. The running times are obtained 
on a Sun SPARCstation-20. 

4.3 Air- Traffic Conflict Resolution 

As a final example, consider an air-trafhc conflict resolution system from [26] 
(see Figure 7). Two aircraft fly towards each other at a fixed altitude and 90 
degree relative orientation. When the distance between the aircraft decreases to 
seven miles, they initiate an avoidance maneuver: each turns 90 degrees to its 






if X2 < kz 






142 Th.A. Henzinger et al. 




Fig. 7. Aircraft collision avoidance protocol 



right, and starts following a half circle. After the half circle is complete, each 
again turns 90 degrees to its right to continue on the original heading along a 
straight path. 

We model this protocol directly as a three-location hybrid automaton with 
the original kinematics. In contrast, the protocol would need to be approximated 
in HyTech in order to be verified. Our model works in a relative coordinate 
system, so that Xr and yr give the position of airplane 2 relative to airplane 1, and 
'ipr gives the angular orientation of airplane 2 relative to airplane 1. In relative 
coordinates, the kinematic equations of this system are 

Xr = —Vi + V2 cos Ipr + ^iVr , Vr = V2 Sin Ipr — ljJ\Xr , — W 2 , (2) 

where v\ (respectively V 2 ) is the airspeed of airplane 1 (respectively airplane 2) 
and u>i (respectively C 02 ) is the angular velocity of airplane 1 (respectively air- 
plane 2). Our automaton has three locations: cruisei, avoid, and cruise2. In lo- 
cation cruisei the airplanes follow straight-line trajectories, with airspeeds vi 
and V 2 in the range [.8, 1]. When the distance between the airplanes decreases to 
seven miles, the control location changes to avoid. On changing to location avoid, 
the heading of each aircraft decreases instantaneously by ^ radians. In location 
avoid, = W2 = 1 and vi = V 2 = 1, so that both airplanes follow circular 
trajectories of the same radius at the same airspeed. When the airplanes have 
completed their half-circles, the location changes to cruise2. Again the heading of 
each aircraft decreases instantaneously by ^ radians, and the airplanes continue 
in straight-line trajectories, with airspeeds vi and V 2 as in location cruisei. Using 
this model, we are able to verify in HyperTech that the two airplanes never 
come within five nautical miles of each other. 
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Abstract. The algorithmic approach to the analysis of timed and hybrid 
systems is fundamentally limited by undecidability, of universality in the 
timed case (where all continuous variables are clocks), and of emptiness in 
the rectangular case (which includes drifting clocks). Traditional proofs 
of undecidability encode a single Turing computation by a single timed 
trajectory. These proofs have nurtured the hope that the introduction 
of “fuzziness” into timed and hybrid models (in the sense that a system 
cannot distinguish between trajectories that are sufficiently similar) may 
lead to decidability. We show that this is not the case, by sharpening 
both fundamental undecidability results. Besides the obvious blow our 
results deal to the algorithmic method, they also prove that the standard 
model of timed and hybrid systems, while not “robust” in its dehnition 
of trajectory acceptance (which is affected by tiny perturbations in the 
timing of events), is quite robust in its mathematical properties: the 
undecidability barriers are not affected by reasonable perturbations of 
the model. 



1 Introduction 

The main limitations of the algorithmic method for analyzing timed and hy- 
brid systems find their precise expression in two well-publicized undecidabil- 
ity results. First, the universality problem for timed automata (does a timed 
automaton accept all timed words?) is undecidable [AD94]. This implies that 
timing requirements which are expressible as timed automata cannot be model 
checked. Consequently, more restrictive subclasses of timing requirements have 
been studied (e.g., Event-Clock Automata [AFH94], Metric Interval Temporal 
Logic [AFH96], Event-Clock Logic [RS99]). Second, the emptiness/reachability 
problem for rectangular automata (does a rectangular automaton accept any 
timed word, or equivalently, can a rectangular automaton reach a given loca- 
tion?) is undecidable [HKPV95]. While several orthogonal undecidability re- 
sults are known for hybrid systems, it is the rectangular reachability problem 
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which best highlights the essential limitations of the algorithmic approach to 
systems with continuous dynamics. This is because the rectangular automaton 
model is the minimal generalization of the timed automaton model capable of 
approximating continuous dynamics (using piecewise linear envelopes). It fol- 
lows that rectangularity as an abstraction is insufficient for checking invariants 
of hybrid systems, and further loss of information is necessary (e.g., initialization 
[HKPV95], discretization [HK97]). 

Both central undecidability results have been proved by encoding each com- 
putation of some Turing-complete machine model as a trajectory of a timed 
or hybrid system. The encodings are quite fragile: given a deterministic Turing 
machine M with empty input, one constructs either a timed automaton that 
rejects the single trajectory which encodes the halting computation of M (ren- 
dering universality undecidable) , or a rectangular automaton that accepts that 
single trajectory (rendering emptiness/ reachability undecidable). However, if the 
specified trajectory is perturbed in the slightest way, it no longer properly en- 
codes the desired Turing computation. This has led researchers to conjecture 
[Fra99] that undecidability is due to the ability of timed and hybrid automata 
to differentiate real points in time with infinite precision. Consequently, one 
might hope that a more realistic, slightly “fuzzy” model of timed and hybrid 
systems might not suffer from undecidability. ^ In a similar vein, in [GHJ97] it is 
conjectured that unlike timed automata, robust timed automata, which do not 
accept or reject individual trajectories but bundles (“tubes”) of closely related 
trajectories, can be complemented. 

In this paper, we refute these conjectures. In doing so, we show that the 
sources of undecidability for timed and hybrid systems are structural, robust, 
and intrinsic to mixed discrete-continuous dynamics, rather than an artifact of 
a particular syntax or of the ability to measure time with arbitrary precision. 
We redo both undecidability proofs by encoding each Turing computation not 
as a single trajectory but as a trajectory tube of positive diameter. This requires 
considerable care and constitutes the bulk of this paper. As corollaries we obtain 
the following results: 

Robust timed and rectangular automata Robust automata introduce 
“fuzziness” semantically, by accepting tubes rather than trajecto- 
ries [GHJ97]. We prove that universality is undecidable for robust 
timed automata (since emptiness is decidable, it follows that they are not 
complementable), and that emptiness/reachability is undecidable for robust 
rectangular automata. 

Open rectangular automata Open automata introduce “fuzziness” syntac- 
tically, by restricting all guard and differential-inclusion intervals to open 

^ Note that “fuzziness,” as meant here, is fundamentally distinct from “discretiza- 
tion,” which is known to lead to decidability in many cases. Intuitively, fuzziness 
preserves the density of the time domain, while discretization does not. Mathemati- 
cally, discretization is performed with respect to a fixed real e > 0 representing finite 
precision, while fuzziness quantifies over e > 0 existentially. 




Robust Undecidability of Timed and Hybrid Systems 147 



sets. We prove that emptiness/reachability is undecidable for open rectan- 
gular automata. The universality problem for open timed automata is, to 
our knowledge, still open. 

A main impact of these results is, of course, negative: they deal a serious 
blow to our ability for analyzing timed and hybrid systems automatically, much 
more so than the previously known results, which rely on questionable, “frag- 
ile” modeling assumptions (one trajectory may be accepted even if all slightly 
perturbed trajectories are rejected, and vice versa). There is, however, also a 
positive interpretation of our results: they show that the “standard” model for 
timed and hybrid systems, with its fragile definition of trajectory acceptance, 
does not give rise to a fragile theory but, on the contrary, is very robust with 
respect to its mathematical properties (such as decidability versus undecidabil- 
ity). For further decidability /undecidability results about the standard model of 
hybrid systems, we refer the reader to [AMP95,BT99]. 

2 Trajectories, Tubes, and Hybrid Automata 

In this paper, we consider finite trajectories only. A trajectory over an alphabet S 
is an element of the language {S x M+)*, where M'*' stands for the set of positive 
reals excluding 0. Thus, a trajectory is a finite sequence of pairs from S x K+. 
We call the first element of each pair an event, and the second element the time- 
gap of the event. The time-gap of an event represents the amount of time that 
has elapsed since the previous event of the trajectory. For a trajectory r, we 
denote its length (i.e., the number of pairs in t) by len(r), and its projection 
onto E* (i.e., the sequence of events that results from removing the time-gaps) 
by untime (t). We assign time-stamps to the events of a trajectory: for the z-th 
event of r, the time-stamp is defined to be tT{i) = where 6j is the 

time-gap associated with the j-th event of r. 

Metrics on trajectories. Let the set of all trajectories be denoted Traj. Assum- 
ing that trajectories cannot be generated and recorded with infinite precision, 
in order to get an estimate of the amount of error in the data that represents 
a trajectory, we need a metric on Traj. Here we define, as an example, one par- 
ticular metric d; in [GHJ97], it is shown that all reasonable metrics define the 
same topology on trajectories. Given two trajectories r and t' , we define: 

— = oo if untime(r) yf untime(r'); 

— = max{|G(*) — G'(0l : 1 < z < len(T)} if untime(r) = untime(r'). 

Thus, only two trajectories with the same length and the same sequence of events 
have a finite distance, and finite errors may occur only in measuring time. The 
metric measures the maximal difference in the time-stamps of any two corre- 
sponding events: two timed words are close to each other if they have the same 
events in the same order, and the times at which these events occur are not very 
different. For instance, for n = (a, l)(a, l)(a, 1) and T 2 = (a, 0.9)(a, 1.2)(a, 1.2), 
we have d(ri,T 2 ) = 0.3. 
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Given a metric, we use the standard definition of open sets. Formally, for the 
metric d, a trajectory r, and a positive real e G K"*", define the d-tube around 
T of diameter e to be the set F(t, e) = {r' : d{r,T') < e} of all trajectories at 
a d-distance less than e from t. A d-open set O, called a d-tube, is any subset 
of Traj such that for all trajectories t G O, there is a positive real e G K’*' with 
T{t,€) C O. Thus, if a d-tube contains a trajectory r, then it also contains all 
trajectories in some neighborhood of r. Let the set of all d-tubes be denoted 
Tube. 

Fhom trajectory languages to tube languages. A trajectory language is any 
subset of Traj; a tube language [GHJ97] is any subset of Tube. Every trajectory 
language L induces a tube language [L], which represents a “fuzzy” rendering 
of L. In [L] we wish to include a tube iff sufficiently many of its trajectories 
are contained in L. We define “sufficiently many” as any dense subset, in the 
topological sense. 

For this purpose we review some simple definitions from topology. A set S 
of trajectories is closed if its complement = Traj — S' is open. The closure S 
of a set S of trajectories is the least closed set containing S, and the interior 
S*”‘ is the greatest open set contained in S. The set S' of trajectories is dense 
in S iff S C S'. Formally, given a trajectory language L, the corresponding tube 
language is defined as [L] = {O G Tube : O C L}. Thus, a tube O is in [L] if 
for each trajectory t G O there is a sequence of trajectories with limit t such 
that all elements of this sequence are in L. Equivalently, L must be dense in O; 
that is, for every trajectory t G O and for every positive real e G M’*', there is a 
trajectory t' G L such that d{r,T') < e. Since the tubes in [L] are closed under 
subsets and union, the tube language [L] can be identified with the maximal 
tube in [L], which is the interior L of the closure of L. 

We will define the semantics of a robust hybrid automaton with trajectory 
set L to be the tube set [L] . This has the effect that a robust hybrid automaton 
cannot generate (or accept) a particular trajectory when it refuses to generate 
(rejects) sufficiently many surrounding trajectories. Neither can the automaton 
refuse to generate a particular trajectory when it may generate sufficiently many 
surrounding trajectories. 

Timed and rectangular automata. An interval has the form (a,b), [a,b], 
{a, b], or [a, b), where a G Q U {— oo}, 6 G Q U {oo}, and a < 6 if / is of the form 
[a,b], and a < b otherwise. We say that the interval / is open if it is of the form 
(a, b), and closed if it is of the form [a, b]. We write Rect for the set of intervals. 

A rectangular automaton [HKPV95] is a tuple A = {S,Q,Qo,Q f,C, E,E\/, 
Init, Pre, Reset, Post, Flow)^, where (i) i7 is a finite alphabet of events; (ii) Q is 
a finite set of locations; (iii) Qo C Q is a set of start locations; (iv) Qf C Q 
is a set of accepting locations; (v) C is a finite set of real-valued variables; (vi) 

^ It is often convinient to annotate locations with variable constraints, so-called in- 
variant conditions. Our results extend straight-forwardly to rectangular automata 
with invariant conditions. 
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E C Q X Q is a finite set of edges; (vii) E\/ : E ^ E is a function that associates 
with each edge e a letter of the alphabet E; (viii) Init : Qo ^ C ^ Rect is a 
function that associates with each start location qo G Qq and variable x G C 
an interval / that contains the possible initial values of this variable when the 
control of the automaton starts in location qo; (ix) Pre : if ^ C — > Rect is a 
function that associates with each edge e and variable x an interval / such that 
the value of x must lie in I before crossing the edge e; (x) Post : E ^ C ^ Rect is 
a function that associates with each edge e and each variable x an interval / such 
that the value of x must lie in / after crossing the edge e; (xi) Reset : if — > 2 *^ 
is a function that associates with each edge e a subset of variables that are 
reset when crossing e; if a variable x belongs to the set Reset(e) then the value, 
after crossing the edge e, of x is taken nondeterministically from the interval 
Post(e,a;); (xii) Flow : Q ^ C ^ Rect is a function that associates with each 
location q and variable x an interval / such that the first derivative of x when 
the control is in location q lies within I. 

Timed automata are a syntactic subset of rectangular automata. A rect- 
angular automaton A is a timed automaton [AD 94 ] if the function Flow of A 
is such that for all locations q G Q, and for all variables x G C, we have 
Flow(g, a;) = [ 1 , 1 ]; that is, every continuous variable is a clock. The timed au- 
tomaton A is open if all intervals used in the functions Init, Pre, and Post are 
open. Similarly, a rectangular automaton A is open if all intervals used in the 
functions Init, Pre, Post, and Flow are open. 

A rectangular automaton A defines a labeled transition system with an infi- 
nite state space S, the infinite set of labels ]&■*■ U E, and the transition relation R. 
Each transition with label ct correspond to an edge step whose event is a G E. 
Each transition with label <5 G ]&■*■ corresponds to a time step of duration < 5 . The 
states and transitions of A are defined as follows. A state {q, x) of A consists of 
a discrete part q G Q and a continuous part x G M". The state space S C Q xMA 
is the set of all states of A. The state (g,x) is an initial state of A if g G Qo 
and X G lnit((7)^. For each edge e = (<71,(72) of A, we define the binary relation 
by (<?i,x) (<72, y) iff X G Pre(e), y G Post(e), and for every coordi- 

nate i G { 1 , . . . ,nj with i ^ Reset(e), we have x^ = y^. For each event a G E, 
we define the edge-step relation by si S2 iff si S2 for some edge 

e G E with Ev(e) = a. For each positive real <5 G K"*", we define the binary time- 
step relation by (<?i,x) (<72, y) iff <Zi = <72 and G Flow(gi). The 

transition relation R C S x S is defined by i? = {^®| e G E} U {^“^1 <5 G K'*'}. 

Trajectory acceptance and reachable locations. We now define the tra- 
jectory language and the reachable locations of a rectangular automaton A. A 
run of the automaton A is a finite path (<70, xq) (<7o>yo) (<7i)Xi) 

(<?i)yi) ■ • ■ (<7n+i7X„+i) in the transition system of A that alternates be- 

tween time steps and edge steps. The run is initial if qo G Qo and xq G lnit(r7o), 
and accepting if <7„ G Q/. The trajectory r = {ao,So){o'i,Si) . . .{an,Sn) is ac- 
cepted by the rectangular automaton A if A has an initial and accepting run 



® To simplify notations, we note x G Init(g) instead of x G lnit(g, a;). 
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Class of Automata 


Emptiness /Reachability 


Universality 


Timed Automata [AD94] 


Decidable 


Undecidable 


Rectangular Automata [HKPV95] 


Undecidable 


Undecidable 



Fig. 1. Known decidability and undecidability results for timed/rectangular automata. 
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Fig. 2. The timed automata Ai, A 2 , and A 3 . 



(go,xo) (go,y) (gi,xi) ... (g„+i,x„+i). The trajectory t 

leads to location qn+i- A location g of A is reachable if there exists an trajec- 
tory r accepted by A that leads to q. We denote by L{A) the set of trajectories 
accepted by A. 

The trajectory-emptiness problem for a rectangular automaton A is to de- 
cide whether or not L{A) is empty. The trajectory-universality problem for a 
rectangular automaton A is to decide whether or not L(A) contains all trajec- 
tories over the alphabet S. The location-reachability problem for a rectangular 
automaton A is to decide if a given location of A is reachable. Note that the 
trajectory-emptiness problem for a class of rectangular automaton is decidable 
iff the location-reachability problem is decidable. The previously known results 
about these problems are summarized in the table of Figure 1. 

Tube acceptance and robustly reachable locations. The rectangular au- 
tomaton A accepts the set [L{A)\ of tubes [GHJ97]. The following examples il- 
lustrate tube acceptance. First, consider the timed automaton Ai of Figure 2(a). 
This automaton accepts all trajectories over the unary alphabet {a} which con- 
tain two consecutive a events with a time-gap in the open interval (1,2). This 
property is invariant under sufficiently small perturbations of the time-stamps. 
Hence the automaton Ai accepts precisely those tubes that consist of trajec- 
tories in L(Ai), and the maximal accepted tube is L(Ai) itself. In the timed 
automaton A 2 of Figure 2(b), the open interval (1,2) is replaced by the closed 




Robust Undecidability of Timed and Hybrid Systems 151 



Class of Automata 


Robust Emptiness/Robust Reachability 


Robust Universality 


Timed Automata 


Decidable 


Undecidable 


Rectangular Automata 


Undecidable 


Undecidable 



Fig. 3. Decidability results about robust timed and rectangular automata. 



interval [1,2]. This changes the set of accepted trajectories but not the set of 
accepted tubes: L{Ai) C L{A 2 ) but [L{Ai)] = [L(^ 2 )j- Notice that the “bound- 
ary trajectories” accepted by A^, with two consecutive a’s at a time-gap of 1 
or 2 but no consecutive a’s at a time-gap strictly between 1 and 2 , are not ac- 
cepted robustly, because there are arbitrarily small perturbations that are not 
acceptable. 

Let us now define the notion of robust reachability. A location g of a rect- 
angular automaton A is robustly reachable if there exists a tube O accepted by 
A such that each trajectory in O leads to q. The automaton A 3 of Figure 2(c) 
illustrates this notion: the locations go, 92 , and ga are robustly reachable, while 
the location gi is not robustly reachable. 

The robust-emptiness problem for a rectangular automaton A is to decide 
whether or not [L(A)j is empty. The robust-universality problem for a rectangular 
automaton A is to decide whether or not [L(A)j contains all tubes over S. The 
robust-reachability problem for a rectangular automaton A is to decide, given 
a location g of A, if g is robustly reachable. In the following sections of this 
paper, we will sharpen the known undecidability results about timed and hybrid 
systems. We will show that the introduction of fuzziness into timed and hybrid 
models via the notion of tubes (this fuzziness can be intuitively seen as the 
semantic removal of equality) does not change the undecidability results. Our 
results are summarized in the table of Figure 3; only the positive result was 
previously known [GHJ97]. 

Some properties of robust timed automata. We recall some results pre- 
sented in [GHJ97]. We will need these notions to establish our results. The first 
proposition tells us that when we consider tube acceptance, we can restrict our 
attention either to closed or to open timed automata. 

Proposition 1. For every timed automaton A, we can construct a timed au- 
tomaton A, called the closur e of A , whose Pre, Post, Init functions use only closed 
intervals, such that L{A) = L{A). Furthermore, we can construct an open timed 
automaton A™*, called the interior of A, such that [L(A)j = [L(A“*)j = [L(A)j. 

The following proposition shows that for open timed automata, tube emptiness 
coincides with trajectory emptiness. 

Proposition 2. For every open timed automaton A and every trajectory t, if t 
is accepted by A along some path, then there is a positive real e G such that 
all trajectories in the tube T{t,c) are accepted by A along the same path. 
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Before defining the tube complement of a timed automaton, we observe an im- 
portant property of the trajectory languages that can be defined by timed au- 
tomata. 

Proposition 3. For every timed automaton A, there is no tube O such that both 
L{A) and its complement L{AY are both dense in O. 

It follows that a tube cannot be accepted by both a timed automaton A and a 
trajectory complement of A. 

For defining the tube complement of a timed automaton A, it is not useful 
to consider the boolean complement Tube — [T(^)] of the tube language [L(A)]. 
For [T(^)] is closed under subsets and union. Therefore, unless [T(^)] = 0 or 
[L(A)] = Tube, the boolean complement Tube— [T(A)] cannot be induced by any 
trajectory language and, hence, cannot be accepted by any timed automaton. 
Thus, for every tube language £ C Tube, we define the tube complement of C to 
be the set 

£" = {OGTube:On|J/: = 0} 

of tubes that are disjoint from the tubes in C. The following proposition shows 
that for every timed automaton A, the tube complement [T(A)]° is induced by 
the trajectory complement L{Ay-, that is, [L{Ay] = [L{A)Y. 

Proposition 4. If L is a trajectory language and there is no tube O such that 
both L and are dense in O, then [LY = [LY- 

For two timed automata A and B, we say that i? is a tube complement of A iff 
B accepts precisely the tubes that do not intersect any tube accepted by A] that 
is, [L{B)] = [L{A)Y- From Propositions 3 and 4, it follows that every trajectory 
complement of a timed automaton is also a tube complement (the converse is 
generally not true) . Since [L{A)Y = in order to construct 

tube complements, it would suffice to construct trajectory complements of open 
timed automata.^ This, however, is not possible, as we show in the next section. 

3 The Robust-Universality Problem for Timed Automata 

In this section, we show that the halting problem for two-counter machines can 
be reduced to the robust-universality problem for timed automata. A two-counter 
machine M is a triple ({6i, . . . , 6„}, C, D), where {bi, . . . , 6„} are n instructions, 
and C and D are two counters ranging over the natural numbers. Each instruc- 
tion bi, 0 < i < n, has one of the three possible forms: (i) a conditional jump 
instruction tests if a counter is 0 and then jumps conditionally to the next in- 
struction; (ii) an increment/decrement instruction increments or decrements the 
value of one of the two counters and then jumps nondeterministically to one of 

^ Similarly, since [L{A)Y = [L{A)Y = [L{AY\, it would suffice to construct trajectory 
complements of closed timed automata. This, however, is known to be impossi- 
ble [AD94]. 
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two possible next instructions; (iii) a stop instruction puts an end to the machine 
execution. A configuration of a two-counter machine M is a triple 7 = (i,c,d), 
where i is the program counter indicating the current instruction, and c and 
d are the values of the counters C and D. A computation of M is a finite or 
infinite sequence 7 = 7071 ... of configurations such that 70 = (0,0,0), i.e. the 
first instruction is 6o> and the initial value of the two counters C and D is 0, and 
for every 7^+1 is a M-successor configuration of 7^, for every z > 0. If 7 is finite 
then its last configuration contains a stop instruction. The halting problem for a 
two-counter machine M is to decide whether or not the execution of M has at 
least one computation that ends in a stop instruction. The problem of deciding 
if a two-counter machine has a halting computation is undecidable. 

Trajectory encoding of a two-counter machine computation. We re- 
view how the undecidability of the universality problem for timed automata was 
established by Alur and Dill [AD94] and explain why their proof does not trans- 
late to the robust-universality problem. Given a two-counter machine M, the 
set of trajectories is defined as follows: {a, 5) G iff (i) a = 

such that {io, cq, do), {h, ci, di), . . . {im, Cm,dm) 
is a halting computation of M; (ii) for all j > 0, the time-stamp of bi^ is j; (iii) 
for all J > 1, (a) if c^+i = Cj, then for every c with time-stamp t in the interval 
(j, j -b 1) there is a c with time-stamp t 1; (b) if Cj+i = Cj 1, then for every 
c with time-stamp t in the interval (j -I- 1, j -I- 2), except the last one, there is a 
c with time-stamp t — 1; (c) if c^+i = Cj — 1, then for every c with time-stamp t 
in the interval (j,j 1), except the last one, there is a c with time-stamp t 1; 
and (iv) the same requirements hold for d’s. Then is nonempty iff M 

has a halting computation. Furthermore, there exists a timed automaton that 
accepts exactly the trajectories not in It follows that the universality 

problem for timed automata is undecidable. 

Note that the z-th configuration is encoded in the interval [z, z-l- 1). To enforce 
the requirement that the number of c events in two successive configurations is 
the same, every c in the first interval has a matching c at the exact distance 1, 
and vice versa. This use of punctuality constraints has the following consequence. 

Proposition 5. Let M be a two-counter machine, there is no tube O G Tube 
such that O is dense in that is, = 0. 

This has nurtured some hope that, by removing the possibility to specify punc- 
tuallity constraints, timed automata might have a decidable robust-universality 
problem. Unfortunately this is not the case. We next show that we can define 
a set of trajectories which forms a tube and encodes halting compu- 

tations of the given two-counter machine M . Furthermore the tube complement 
of this tube language can be defined by a robust (open) timed automaton. The 
undecidability of the robust-universality problem and the nonclosure under com- 
plement of robust timed automata will follow. 

Tube encoding of a two-counter machine computation. To facilitate 
the definition of the undecidable tube language, we first introduce 
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some new notions. We call an open (closed) slot an open (closed) interval of 
the real numbers. We define the open (closed) slot between ti and t2 as the set 
{t \ ti < t < 12} (respectively, {t \ ti < t < ^2})- Given two real numbers ti and 
t2 with ti < t2, we say that (tsjG) (respectively [t^,t4\) is the open (closed) slot 
generated by t\ and t2 if both t\ + 1 = t^ and ^2 + 1 = G- 

The main idea of Lj"^|‘^(M) is that we encode the configuration i within the 
open interval (z, z+1), and the next configuration z+1 will be encoded in the open 
slot generated by the time of the beginning and the end of configuration z. For 
the encoding of the elements of a configuration and their relation with the next 
configuration we also use open slots. For instance, we use the triple • E'"®* 

to encode that bj^ is the instruction executed in the z-th configuration; the letters 
and E'"®* are used as delimiters of the instruction, and to generate the slot 
for the next instruction. Let us assume that t\ and ^2 are the time-stamps of B'"®* 
and respectively. Then the encoding of the next instruction has to take place 
in the open slot (G -|- 1, ^2 + 1) generated by the slot for the current instruction. 
As we use a dense time domain, this constraint can always be satisfied. We will 
proceed in the same way for the encoding of the values of the two counters. 
The value of the counters C and D are encoded as follows: if the value of the 
counter (7 is zz in configuration z, then the pair is repeated u times in the 

encoding of the configuration z. If the counter C is unchanged from configuration 
z to configuration z -|- 1 , we verify that the • e'^ sequences in configuration z -|- 1 
appear exactly in the open slots defined by the b'^ •e'^ sequences in configuration z. 

Having the intuition underlying the language *-Tubr(-^)> define it and 

establish that the set of trajectories in Lj"^|'^(M) correspond to a non empty 
set of tubes iff the machine M has a halting computation. The set of events 
that we will use in the encoding is the following: (i) and are the 

delimiters for the beginning and end of the encoding of a configuration; (ii) 
B'"^* and E'"®* are the delimiters for the begin and end of the encoding of the 
instruction executed in a configuration; (iii) 61, 62, • ■ • , are used to represent 
the n instructions; (iv) B^ and E*- are the delimiters for the encoding of the 
value of the counter C in a configuration; (v) B'^ and E'^, for the counter D\ 
(vi) b“^ and are used to encode the value of the counter ( 7 ; (vii) b^ and e"^, 
for D. The trajectories of Lj()^|‘^(M) agree with the following regular expression: 

(BConf , ginst .(bi\b2\...\b„)- • B‘= • (b" • c • e")* • E'^ • B° • (b^ d . s'*)* • E° • E‘=°''f)*. 

Furthermore, if the configuration z contains the sequence B'"^* • bj^ ■ E'"®*, then the 
configuration i + 1 contains the sequence B'"®* • ■ E'"^*, where is a valid 

next instruction of hj^ . The first configuration is encoded in the open interval 
(0, 1); that is, if the event occurs at time t\ and the event occurs at 
time t2, then 0 < ti < t2 < 1 - The configuration z -b 1 is always encoded in the 
open slot defined by the configuration z; that is, if the event of configuration 
z occurs at time t\ and the event occurs at time t2, then the encoding of 

the configuration z -b 1 takes place in the open slot (ti -b 1 , t2 + !)• The encoding 
of the instruction executed during the configuration z -b 1 takes place in the slot 
defined by the encoding of the instruction executed in configuration z; that is, if 
B'"^* and E'"^* appear at times ti and ^2 in encoding of configuration z, then B'"®* 
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and E'"®* appear at times and in the encoding of configuration f + 1 with the 
following (open) real-time constraint: -b 1 < ts < ^4 < ^2 + 1- We only explain 

in details the case when the counter C is incremented from configuration i to 
configuration The other operations are left to the reader. If in configuration 
i the events and E*- occur at times t\ and t 2 , respectively, then the events 
and E*- appear for configuration i + 1 within the open slot (t i -b 1 , t 2 + 1) • For each 
b'^ • e"^ sequence, such that occurs at time ti and occurs at time t 2 , in the 
encoding of configuration i, there is exactly one sequence b*^ • e*^ sequence in the 
encoding of configuration f -b 1 that takes place in the open slot (ti -b 1, ^2 + !)• 
Conversely, each b'^ • e'^ that appears in the encoding of the configuration f -b 1, 
with the exception of the last, must lie in the open slot defined by the b*^ • e*^ 
sequence of configuration i. This requirement is noted RTg. Finally, the last b'^-e'^ 
sequence in the encoding of configuration z -b 1 appears in the slot generated by 
the two events B^ and E^ if C = 0 in configuration z, and appears in the slot 
generated by the last e'^ event and E*- event of configuration z if C > 0 in that 
configuration. 

The following proposition is a direct consequence of the use of strict inequal- 
ities in the definition of the language 

Proposition 6. Let M he a two-counter machine, for every trajectory t\ that 
belongs to there exists a real e > 0 such that for every trajectory T 2 , 

if d{Ti,T 2 ) < e then T 2 € (M). 

Corollary 1. For every two-counter machine M with a halting computation, 
is a nonempty tube language. 

Corollary 2. There is no tube O that is dense both in LVubr(^) 

Note also that by Proposition 6 and Corollary 2, we know that the tube se- 
mantics of a timed automaton that accepts the complement of the trajectories 
of is exactly the complement of the tube language The 

following lemma shows that it is possible to construct such a timed automaton. 

Lemma 1. There exists a timed automaton Am that accepts exactly the trajec- 
tories that are not in 

Proof. It is sufficient to show that for each of the requirements defining 
we can construct a timed automaton that accepts exactly the trajectories that 
violate the requirement. The union of these automata is exactly what we are 
looking for: the timed automaton that accepts the trajectory complement of 
i-Tubr(-^)- Flue to the lack of space, we just give here the automaton for the 
complement of requirement RTg; the other requirements can be found in [HR99]. 
The timed automata for requirement RTg is shown in Figure 4. This automaton 
accepts exactly the trajectories which contain two adjacent configurations z and 
z -b 1 such that (i) the instruction executed in configuration z increments the 
counter C, that is b G , where is the subset of instructions that increment 
the counter C; (ii) there is a sequence b*^ • e'^ in configuration z that defines an 
open slot in configuration z -b 1 which does not contain the sequence b*^ • e'^. □ 
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Fig. 4. A timed automaton for the negation of requirement RT§ 



Combining Lemma 1 and Proposition 4, we obtain the following theorem. 

Theorem 1. For every two-counter machine M , there exists a timed automa- 
ton Am that accepts every tube iff the two-counter machine M has no halting 
computation. 

Corollary 3. The robust-universality problem for timed automata is undecid- 
able. 

As the robust-emptiness problem for timed automata is decidable, we obtain the 
following: 

Corollary 4. There is a tube language definable by a timed automaton whose 
tube- complement is not definable by a timed automaton. 

From these results we can derive the following result about the trajectory lan- 
guages of open timed automata (already established in [Her98]): 

Theorem 2. There is a trajectory language definable by an open timed automa- 
ton which trajectory-complement is not definable by a timed automaton (open or 
not). 

Proof. By reductio ad absurdum. We have constructed a timed automaton Am 
that accepts the complement of the trajectories contained in This 

automaton Am defines a set L{Am) of trajectories such that [L{Am)] is exactly 
the tube complement of [LYubr(M)]. By Proposition 1, there exists an open timed 
automaton, namely, the interior of Am, denoted A^ff, such that [L{A^ff)] = 
[L{Am)] = [LV"be*^(M)]'^. By Lemma 4, if we were able to complement the open 
automaton , then we could obtain an automaton whose tube semantics would 
be This, however, is impossible, as the robust-emptiness problem of 

timed automata is decidable, which would allow us to decide the halting problem 
for two-counter machines. □ 
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4 The Robust-Reachability Problem for Rectangular 
Automata 

In this section we investigate undecidable reachability problems and show that 
they remain undecidable even when we remove equality from the specification 
formalism. In [HKPV95], it is shown that the formalism of rectangular automata 
lies at the boundary between decidable hybrid formalisms and undecidable ones. 
We show here that this boundary stays valid if we do not use equality. For this 
purpose, we use another tube encoding of two-counter machines computations. 
With each halting computation {io,co, do), {ii, ci, di), . . . , {in, Cn,dn), we asso- 
ciate the tube 



with 0 < j < n and the following timing constraints. We just give the constraints 
for the encoding of the value of counter C; the same requirements hold for the 
counter D. Initially the value of the counter C is zero. To encode C = 0, we 
require that if the events and e*^ are issued at times t\, t 2 , and then 

the following constraint is satisfied: ti + \ < < t\ + 1. Let d\ denote the 

distance that separates the events and b'^, and let d 2 denote the distance that 
separates the events B*- and e'^ in the encoding of the value of C in configuration 
i. In the same way, let do and d^ be those two distances in the encoding of the 
value of C in configuration i + 1. Then we have the following requirements: (a) 
if C is incremented between i and i + 1, then ^ < da < ^4 < ^; (b) if C is 
decremented between i and i+1, then 2d\ < do < d 4 < 2d2] (c) if C is unchanged 
between i and z-l- 1, then d\ < do < d^ < d 2 - We denote this trajectory language 
L8plrRect(A^). 

Lemma 2. The trajectory language LopenRect(-^) definable by an open rect- 
angular automaton Am- 

Proof. We sketch the proof by giving an open rectangular automaton to in- 
crement the counter C. The automaton is given in Figure 6. To see that the 
automaton checks exactly the desired constraints, we first establish bounds on 
the values of the variables x and y at times to, t\, t 2 , and to represented in 
Figure 7. The bounds are given in the table of Figure 5. So at time to, we have 
X e (di,-hoo) and y G (—00,^2)- Now let us see the constraints that we obtain 
on do and d^. First, by taking into account that x G (di, -l-oo) at to and the flow 
of X in qo in included in the interval (—2, 0), we can deduce that do G (^,-hoo). 
Second, by taking into account that y G (—00, d2) at to and that the flow of y 
in qo is included in the interval (—00,— 2), we obtain do G (—00, ^). As b*^ is 
issued before e"^, we have do < d^, and thus ^ < do < d^ < ^, as desired. □ 

As a direct consequence of the last lemma, we have the following. 

Theorem 3. The trajectory- emptiness and location-reachability problems for 
open rectangular automata are undecidable. 
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Fig. 5. Inferior and superior bounds on the values of variables x and y. 




FIow(<3'2 ,x) G ( 1 , 2 ), Flow(g 2 , y) ^ (0, 1) 

Flow(g 3 , ai) G (0, 1), Flow(g 3 , y) G (0, 1) 
Flow(g 4 , x) G (0, 1), Flow(g 4 , y) G (—1, 0) 
Flow(g 5 ,ai) G ( — 2,0), Flow(g 5 ,y) G ( — oo, — 2) 

Flow(g6,y) ^ (-00,-2) 



Fig. 6. Open rectangular automaton to check incrementation of counter C. 
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to ti t2 to t4 to 



Fig. 7. Two successive encodings of the value of counter C. 



The following proposition is a generalization to open rectangular automata of 
Proposition 2. 

Proposition 7. For every open rectangular automaton A and every trajectory t, 
if T is accepted by A along some path, then there is a positive real e G M’*' such 
that all trajectories in the tube T(t, e) are accepted by A along the same path. 

This proposition implies that tube and trajectory emptiness coincide for open 
rectangular automata, so we have the following theorem. 

Theorem 4. The robust-emptiness and robust-reachability problems for rectan- 
gular automata are undecidable. 
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Abstract. In this paper, we present a scheme of stochastic hybrid sys- 
tem which introdnces randomness to the deterministic framework of the 
traditional hybrid systems by allowing the flow inside each invariant set of 
the discrete state variables to be governed by stochastic differential equa- 
tion (SDE) rather than the deterministic ones. The notion of embedded 
Markov chains is proposed for such systems and some illustrative exam- 
ple from high way model is presented. As an important application, these 
ideas are then applied to the state space discretization of one dimensional 
SDE to obtain the natural discretized stochastic hybrid system together 
with its embedded MC. The invariant distribution and exit probability 
from interval of the MC are studied and it is shown that they converge 
to their counterparts for the solution process of the original SDE as the 
discretization step goes to zero. As a result, the discretized stochastic 
hybrid system provides a useful tool for studying various sample path 
properties of the SDE. 



1 Introduction 

In the conventional formulation of hybrid system (See, for example, [6]), there is 
no place for randomness. Although the deterministic framework captures many 
characteristics of the real systems in practice, in other cases, the missing flavor of 
randomness will indeed be a fatal flaw because of the inherent uncertainty in the 
environment of most real world applications. The idea of introducing stochastic 
hybrid system is not new. Different researchers have tried to propose different 
models from their own perspectives. For the most recent and relevant literature, 
the readers are referred to [1,5,10,2,11,9]. The most important difference lies in 
where to introduce the randomness. 

One obvious choice is to replace the deterministic jumps between discrete 
states by random jumps governed by some prescribed probabilistic law. Hence 
the evolution of the discrete states constitutes a time homogeneous Markov 
chain. The question remained then is when does such jump occur? In [1], the 
jumps occur every e time, and the effect when e — > 0 is studied. In [10], however, 
the transitions follow a continuous time Markov process. In both papers, the 
discrete random transitions are assumed to be independent of the continuous 
dynamics, therefore the models can actually be better viewed as an extension of 
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Markov process with some continuous states attached whose evolutions follow 
state-dependent deterministic differential equations. 

Another choice is to replace the deterministic dynamics inside the invariant 
set of each discrete state by a stochastic differential equation (SDE) . Therefore, 
even if we keep the deterministic discrete transition part, starting from a fixed 
initial state, different guards can be activated depending on the realization of 
the solution stochastic process, thus different discrete transitions occur randomly. 
More general models can be proposed by blending the above two choices. 

This paper is organized as following: in Section 2, we will try to give a general 
definition of stochastic hybrid system based on the second choice mentioned 
above. An example will be shown in Section 3 together with its analysis. In 
Section 4, the idea will be applied to a more general problem, in which we will 
approximate the solution of the SDE in by the stochastic hybrid automata 
obtained from state space discretization. And finally we will discuss the special 
case of gradient system in the last section. The proofs of the theorems are not 
included due to the limit of space and will appear in subsequent paper. 



2 General Definition 

Definition 1 (Stochastic Hybrid System). A stochastic hybrid system (or 
automata) is a collection H = (Q, X,Inv, f, g,G, R) where 

— Q is a discrete variable taking countably many values in Q = {q\, q 2 , - ■ • }; 

— X is a continuous variable taking values inX = for some N G N; 

— Inv : Q ^ 2^ assigns to each q G Q an invariant open subset ofX; 

— /, 5 : Q X X — > EX are vector fields; 

— G : E = Q X Q ^ 2^ assigns to each e G E a guard G(e) such that 

• For each e = {q,q') G E, G(e) is a measurable subset ofdInv{q) (possibly 
empty); 

• For each q G Q, the family {G{e) : e = {q,q') for some q' G Q} is a 
disjoint partition of dlnv{q) . 

— R : E X ~K ^ 'PPQ assigns to each e = {q,q') G E and x G G{e) a 
reset probability kernel on X concentrated on Inv{q'). Here E(X) denote the 
family of all probability measures on X. Furthermore, for any measurable set 
A Clnv{q'), R{e,x){A) is a measurable function in x. 



Remark 1. The measurability assumption on R in the preceding definition is 
made to ensure that the events we encounter later are measurable w.r.t. the 
underlying cr-field, hence their probabilities make sense. 



Definition 2 (Stochastic Execntion). A stochastic process (X(t),Q(t)) G 
X X Q is called a stochastic execution iff there exists a sequence of stopping 
times To = 0 < Ti < T 2 < • • • such that for each n G N, 
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~ In each interval [r„,Tn+i), Q{t) = Q(r„) is constant, X{t) is a (continuous) 
solution to the SDE: 

dX(t) = f{Q{Tn),X{t))dt + g{Q{Tn),X(t))dBt , 

where Bt is the standard Brownian motion in M; 

~ Tn+I = inf{t > Tn : x{t) ^ Inv{Q{Tn))}; 

- X{T~_^_^) e G(Q(t„),(5(t„+i)) where X{r~^-y) denotes X(t); 

— The probability distribution of X (tu+i) given X(T~_f_i) is governed by the law 
R{e„,X{T~_^_■^^)), where e„ = (Q(t„), Q(t„+i)) G E. 

Definition 3 (Embedded Markov Process). In the notation of the previous 
definition, define Qn = Q{t„), X„ = X{Tn)- Then {(Qn, Xn),n > 0} is called 
the embedded Markov process for the stochastic execution {X{t),Q{t)). 

Under these definitions, for example, a typical stochastic execution starts 
from (Qo 7 -^o) and the continuous state X{t) evolves according to the SDE 

dX{t) = /(go, X{t)) dt + g{Qo, X{t)) dBt, X(0) = Xq 

until time ti when X{t) first hits dInv{Qo). Then depending on the hitting 
position X{t(~), (say, X{Tf) G G(e) where e = (go,gi) for some Q\ G Q), the 
discrete state jumps to Q{ti) = Q\ and the continuous state is reset randomly to 
X{ti) = Xi according to the conditional probability distribution R{e, X{t(~)){-) 
and the same process is repeated with (Qi,Xi) replacing (go,^o) and so on. 

Lemma 1. {{Qn, Xn)} defined above is indeed a Markov process with transition 
probability: 

P{Qn+i = q',Xn+i & dx'\Qn = q,Xn = x) = [ R{e,y){dx') P{Y„;{r]) = dy) , 

Jy&G(e) 

( 1 ) 



where e = {q,q'), Y^ft) is the solution to the SDE 

dY (t) = f{q, Y {t)) dt + g{q, Y (t)) dBt, y(0) = a; , 
and rj = inf{t > 0 : Y^ft) ^ Inv{q)} is the first escape time ofY^ft) from Inv{q). 

Lemma 2. If the reset kernel R{{q,q'),x) = R{q') does not depend on q nor x, 
then {Qn} itself is a Markov chain (MC) with transition probability (n > 1): 

P{Qn+i = q\Qn = q)= f P{Yx{v) G G{e))R{q){dx) , (2) 

J xGlnv(q) 

where e,Yx,r] is defined in the previous lemma. For n = 0, the transition proba- 
bility depends on the initial distribution of X{0). 
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Remark 2. The condition in Lemma 2 is fairly restrictive and excludes many 
general stochastic hybrid systems. The point of imposing this condition is to 
make calculation tractable. Furthermore, as we will see in the later sections, this 
special class of systems is still rich enough to admit many important applications. 

The reason we introduce the embedded MC is that in most cases, it is hard 
if not impossible to get an explicit expression of the stochastic execution for a 
stochastic hybrid system. If all we are interested in is the reachability analysis of 
the discrete states transitions, then {Qn} will capture all the necessary informa- 
tion. This is the case if a subset of the discrete states is defined to be the “bad” 
states and a controller is designed to minimize the probability of reaching these 
states within a given time horizon. Or alternatively, some states are defined to be 
safe and we want to maximize the probability that the execution will remain in 
these states for as long as possible. At first sight these observation does not seem 
to be applicable in general, since in most cases, the definition of bad states and 
safe states involve both the discrete and continuous states. However, by breaking 
up the corresponding invariant sets and adding more discrete states and trivial 
reset kernels, we can always reduce the original system to a new one satisfying 
the above conditions, at least in the case when the support of any reset kernel 
is contained exclusively in safe or bad set. 

3 A Simple Example 

To clarify the above concepts, consider the following simple example. Two cars, 
labeled 1 and 2 with car 2 in the lead, are moving from left to right on a highway 
(see Figure 1). Due to various random factors such as road condition, wind, and 
the presence of human operators, the motions of both cars are stochastic. If 
we absorb all the randomness into the motion of car 1 and ignore the possible 
occurrence of emergency braking, then the motion of car 2 can be modeled as 
having a constant speed V 2 - Let Ax be the distance between the two cars. Let 
do > di > ^2 > ds > 0 be four thresholds. We propose the following hybrid 
control scheme for car 1 (see the diagram in Figure 2): It consists of 3 discrete 
states {1,2,3} corresponding to chasing, keeping and braking respectively. 

1. Chasing: In this stage, Ax > d, 2 , and car 1 will try to catch car 2 at speed 
vi > V 2 - So the perturbed motion of car 1 is governed by x\ = v\ + dBt, 
where Bt is a standard 1-D BM; 



do 



Ax 



di 




Fig. 1. A two-car platoon on the highway 
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2. Keeping: In this stage ds < Ax < di, and car 1 will try to move at V 2 under 
the perturbation dBt] 

3. Braking: If Ax < ds, then car I will brakes according to some prescribed 
procedure until Ax = dg. For simplicity, we ignore the presence of noise 
during braking. 




Fig. 2. Diagram for the stochastic hybrid system 



The invariant sets and guards for each discrete state are also shown in Fig- 
ure 2. The reset kernels are trivial, or more precisely, R(e,x) is concentrated at 
X for any e = (g, q') € E and any x G G{e). It is easily seen that H satisfies the 
condition of Lemma 2 . Hence the successive visits to the discrete states {Qn} is 
a MC. Actually its probability transition matrix is 

0 1 0 \ 
p 0 1 -p , 

10 0 / 




where p = (^2 — dz)/{d\ — ds). The first and third row of P is obvious and the 
second row follows from ([3]): 



Lemma 3. Let Bt be a standard BM starting from 0. For a < 0 < b, define 
Ta = inf{t > 0 : Bt = a}, n = inf{t >0: Bt = b}. Then P{Ta < n) = ^ 
and E{Ta A Tif) = —ab. Here Ta A Tb denotes min(Ta, Tb). 

Calculation shows that the stationary distribution for P is ( 3 ^, 3 ^, 53 ^ )• 
Therefore the fraction of time the system spends in each discrete state is pro- 
portional to: 



ETi ET2 

^3-p’ 3-p’ 



(1-p)AT3 



), 



3-p 
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where ETi = {do — d 2 )/{v\ — W 2 ), ET 2 = {d\ — d 2 ){d 2 — do), ET 3 = to are the 
expected sojourn time in each discrete state respectively. 

In practice, we want to maximize the time the stochastic hybrid system 
spends in the keeping state and minimize the time it spends in the braking state. 
This can be done by adjusting the thresholds do,d\,d 2 ,do properly. Sometimes 
this choice is restricted by other physical constraints. However, we can always use 
more thresholds and thus more complex stochastic hybrid controller to achieve 
the goal within the various physical constraints. This technique will be illustrated 
in the next section. 



4 State Discretization of 1-D Stochastic Differential 
Eqnation 

4.1 Motivation and Definition 

Consider the following stochastic differential equation in M: 

= J{X{t)) + dBt, X(0)=0, (3) 

where / : R ^ K is smooth and dBt is white noise with spectral density 1. 
Define a series of stopping times r„ inductively as: tq = 0, r„ = inf{t > r„_i : 
\X{t) - X{Tn-i)\ = 4. ^ Let Sn = X{Tn). Then {£'„} is a MC 

taking values in i5 • Z. Sn captures many sample path properties of the solution 
process X{t), for example, whether X{t) is recurrent, or less obviously, whether 
X{t) crosses an interval of length less than 6 infinitely many times. 

Define n = sup„{r„ : r„ < t} and let Yt = X{Tt). Then Yt is piecewise 
constant with value Sn in time interval [r„,r„+i). Define Z{t) to be the solution 
process to the stochastic differential equation: 

^ = f{Y,)+dB,. (4) 

Comparing equation (3) and (4) and noticing that during time interval 
[Tn,Tn+i), \X {t) ~ Yt\ < (5 by the definition of r„’s and / is continuous, we 
can expect that as (5 ^ 0, Z{t) approaches X{t) in distribution, hence Z{t) 
is a good approximation to X{t) which is often impossible to calculate explic- 
itly. However, it is still difficult to solve equation (4) since Yt depend on the 



Inv(-l) Inv(l) 

-2*5 -5 0 5 ^ 

^ 

Inv(-2) Inv(O) lnv(2) 



Fig. 3. Discretization of state space 
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original solution process X{t) through t„’s and S'„’s. So to solve equation (4), 
theoretically we still have to solve equation (3) first. 

One way to get out of this loop is to use the fact that X{t) can be approx- 
imated by Z{t), hence t„’s and 5„’s can also be approximated by the corre- 
sponding random variables defined from Z(t). This will lead to the discretized 
stochastic hybrid system (DSHS) defined below. 

Definition 4 (Discretized Stochastic Hybrid System). The discretized 
stochastic hybrid system for equation (3) is H = {Q,X,Inv, f,g,G,R) where 
Q = Z, X = R, and 

— Inv{k) = ((fc — l)i5, {k + l)i5) for any k G Q; 

— f{k, •) = f{kS), g{k, •) = 1 constant functions; 

— G{k, k—1) = {(fc— l)i5}, G{k, A: -1-1) = {(/c-|-l)(5} are singletons and G{k, 1) = % 
for all other 1; 

— Reset kernels are trivial. 

Since H satisfies the condition of Lemma 2, {Qn} defined as in Section 2 is 
a MC. By discussion at the beginning of this section, it is expected that {Qn} 
approximates the MC {S'n} defined from the solution X{t) to equation (3). (In 
the following development, we will use to stress the dependency of H on the 
discretization step d only if necessary). 

Obviously the probability transition matrix Q for {Qn} satisfies: Qij = pi, 
if j = i + 1; Qij = qi = l— Pi if j = i— 1 and Qij = 0 otherwise. Such a chain 
is called a death and birth chain and we will calculate pkS and qk’s as follows: 
The solution to the stochastic differential equation dY ft) = f{kd) dt + dBt with 
initial condition T(0) = kd is Yff) = kd + f{kd)t + Bt, i.e. the BM starting from 
kd and with drift p = f{kd). If we use Bf to denote the BM starting from 0 and 
with drift /i, then 

Pk = P{Bf reaches d before it reaches — <5) . (5) 

So the problem becomes calculating the exit distribution of Bf from {—d, d). We 
will derive the probability in a more general setting. Assume p ^ Q since the case 
when p = 0 has already been considered in Lemma 3. Let a < 0 < b. Denote 
Ta = inf{t >0:Bf = a}, n = inf{t > 0 : = b}. 

Lemma 4. Bf first exits (a, b) from b with probability 

-2/^a _ 1 

p{n<T^) = (6) 

Therefore by taking a = —d, b = d and p = f{kd), we have pk = 4>[df{kd)] 
where (j) is the monotonically increasing function defined by 

231 1 



For a plot of function (j), see Figure 4. 
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Fig. 4. Plot of (j) 



4.2 Recurrence vs. Stability 

Having obtained the probability transition matrix Q, the natural question we will 
ask ourselves is: what is the relation between the deterministic part of differential 
equation (3), i.e. 



- = /(a:), x(0) = 0, (8) 

and the embedded MC {Q„}? For (8) we have notions such as equilibrium and 
various kinds of stability. What are their counterparts for {Qn}? Intuitively if 
equation (8) has a globally stable equilibrium then the sample paths of {Qn} 
should also be centered around the equilibrium most of the time even if the start- 
ing point is far away, thus stability in some probabilistic sense can be expected. 
It turns out that the notions of recurrence and transience are good candidates 
for this. Assume MC {Qn} is irreducible, i.e. starting from any state, there is a 
positive probability of jumping to any other state in finite steps. 

Definition 5 (Recurrent and Transient MC). A MC {Qn} on a countable 
state space S is called recurrent if and only if starting from any state x G S, it 
will return to x in finite time with probability 1, or more precisely, if and only if 

P{Tx < oolQo = a;) = 1 \/x G S , 

where = inf{n > 1 : Qn = a;}. Otherwise {Qn} is called transient. 



Definition 6 (Positive Recurrent MC). A recurrent MC {Qn} on a count- 
able state space S is called positive recurrent if and only if E[Tx\Qq = x] < oo 
for all X G S. 





168 J. Hu, J. Lygeros, and Sh. Sastry 



An important characteristic of a positive recurrence chain is that its invariant 
distribution exists and is unique ([3]). In general, positive recurrence implies 
recurrence, but not the other way around, since symmetric random walk on 
integer grid Z is an example of recurrent but not positive recurrent chain. 

Now consider the MC {Qn} obtained in subsection 4.1. Obviously it is irre- 
ducible. Let {Qn} and {Q~} be the MC’s obtained by observing {Qn} on the 
subset N+ = {0, 1,2, •••} and N“ = {0, — 1, — 2, • • • } respectively. Both {Q^} 
and {Qn} are irreducible. The following lemma justifies our interest in them. 

Lemma 5. {Qn} is (positive) recurrent iff both {Qf} and {Qf} are (positive) 
recurrent respectively. Furthermore, if is the stationary distribution of {Qf} 
on TT~ is the stationary distribution of {Qn} on then tt = a7r+ -|- (1 — 
a)7T“ is the stationary distribution of {Qn} on Z, where 

7T“(0)po 

Oi = . 

7'‘“(0)po + 7>'+(0)go 

Notice that the transition matrix Q+ has the property Q'^{i,j) = 0 when 
\i — j\ > 1, hence it is a death and birth chain. The following lemma is a standard 
result from probability theory (see [3]): 

Lemma 6. {Qf} is recurrent if and only X)m=o Ojli = oo, {Q„} is 
positive recurrent if and only if J2m=oT[jLi Pj-i / Qj < oo (here po = 1). In the 
latter case, the stationary distribution 7t+ of {Q^} is: 



T^^ii ) = n 



Pj-l 



En 

m—O j=l 



Pi-1 

9i 



* = 0 , 1 , 2 , 



Note the products are interpreted as 1 whenever m = 0. 

Similar argument for {Qf} can be established by symmetry. Assembling 
Lemma 5, Lemma 6, equation (7) together, we get 

Theorem 1 (Recurrence of DSHS). The embedded MC {Qn} of the dis- 
cretized stochastic hybrid system of (3) is recurrent if and only if 



1 - exp[-2Sf{jS)] 

^0 fJi - 1 



En 



and 



E n 

m—O j=—m 



exp[2Sf{jS)] - 1 
1 - exp[-2Sf{jS)] 



(9) 



{Qn} is positive recurrent if and only if 






and 



OO — 1 



E n 

m—O j— — m 



1 - + 1)<5)] 

)>[Sf{jS)] 



< OO . 

( 10 ) 



In the latter case, the stationary distribution tt of {Qn} is given by Lemma 5. 
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4.3 Boundary Between Recurrence and Transience 

From Theorem 1, it is evident that whether {Qn} is (positive) recurrent depends 
only on the “tail” of function /, i.e. the asymptotic behavior of f{x) when 
X — > ±oo. In general, we have 

Lemma 7 (Comparison Lemma). Suppose /, (/ : K — > K are two smooth 
vector fields such that 

f{x) > g{x), f{-x) < g{-x) for x sufficiently large, 

Then if {Qn(f)} is (positive) recurrent, so is {Qn{g)}- Conversely, if {Qn{g)} 
is transient, {Qn{f)} is also transient. 

Inspired by [3], let us look at / of the form 

{ Cx~'" X > M 

-C(-x)-” x<-M (11) 

do not care \x\ < M 

for some constant C and r > 0. Note we have deliberately made / to be an odd 
function outside {—M,M) such that the corresponding MC {Q^} and {Q~} are 
mirror image of each other. So by Lemma 5 we need only to consider one of 
them, say, {Qn}- If C < 0, then by the Comparison Lemma and the previous 
paragraph, {Qn} is recurrent, so we assume C > 0 here. 

Proposition 1. Assuming C > 0. The DSHS {Qn} corresponding to f in (11) 
is recurrent if r > 1 or if r = 1 and C < 0.5. {Qn} is transient if r < 1 or if 
r = 1 and C > 0.5. 

Note the above conclusion is independent of the discretization step S. Next we 
will discuss the boundary of positive recurrence. Suppose / is of the form: 

{ —Cx~^ X > M 

C{-x)-^ x<-M (12) 

do not care \x\ < M 

where C, r are positive constants. A similar argument generates: 

Proposition 2. Assuming C > 0. The DSHS {Qn} corresponding to f in (12) 
is positive recurrent if r < 1 or if r = 1 and C > 0.5. {Qn} is not positive 
recurrent if r > 1 or if r = 1 and C < 0.5. 

5 DSHS of Gradient System 

If equation (8) is a gradient system ([8]) of the form: 

dx 
dt 



fix) = -\7V{x) 



(13) 
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SIGl SIG2 

Fig. 5. DSHS for a gradient system 



for some V € C^(M), then each local minimum of V (x) is an equilibrium of (13) 
and in the embedded MC {Qn} of the corresponding DSHS, states in the vicinity 
of each equilibrium constitute an strongly interacting group (SIG) in the sense 
that in any typical execution of {Qn}, once the state jumps into an SIG, it will 
stay inside it for a relatively long period before jumping to another SIG. (See 
Figure 5). In many applications it is often the case that we want to choose some 
suitable control so as to make the system evolve inside some desired valleys for 
as long as possible while avoiding some undesired trap. 

Under this setting, the conclusion of Proposition 1 and Proposition 2 in the 
last subsection translates into: {Qn} is recurrent (transient) if V{x) approaches 
— oo slower (faster) than — iln(|a;|) as \x\ oo respectively; {Qn} is (not) pos- 
itive recurrent if V(x) approaches oo faster (slower) than iln(|a;|) as |x| ^ oo 
respectively. Therefore instead of the clear cut boundary between stability and 
non-stability in the deterministic system, the DSHS have a blurred boundary 
between positive recurrence and transience, with V{x) growing asymptotically 
between — jln(|a:|) and |ln(|a:|) corresponding to recurrent but not positive re- 
current {Qn}- In this subsection, we will always assume that V{x) is chosen such 
that for 5 small enough, the corresponding {Qn} is positive recurrent and hence 
has a stationary distribution tt. We will elaborate on the asymptotic behavior of 
7T as i5 ^ 0 and reveal its relation with V (x). 

From Lemma 5 and Lemma 6, tt can be written as: 7t(z) = aTr~^(i) -I- (1 — 
a)TT~(i) for alH G Z with 



7T (0)^[(5/(0)] 

-(0)<^[5/(0)] + 7r+(0)(l-().[<5/(0)]) 



TT 
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and 

A miij - 1)^)] / V ft muj - 
/i I -mm] ’ 

t 4 l-cj^[Sf{{j + l)5)] l-ct>[6f{{j + l)S)] 

II mus)] / mus)] ’ 

(14) 

This messy-looking expression takes an especially simple form as i5 ^ 0. To 
reveal this, for each i5 > 0 denote the stationary distribution of {Qn} for 
the discretized stochastic hybrid system with discretization step <5. Define 
function : M ^ M as: u^{x) = 7T^{k)/S, if a: G [kS, {k + 1)<5) for some k G Z. 
Then it can be easily checked that satisfies: m^u^{x)dx = 1, and has 
roughly the same shape as tt^ . Therefore the discrete distribution is converted 
to a continuous density function . Moreover, 

Lemma 8. Suppose V (x) is chosen such that exists for i5 > 0 small enough. 
Then 



7r+(i) = 
TT~{i) = 



lim = e-2fo(fo-^(^)l Vx, 2 / G R . 

s^ou^(x) 

We need the following notion to ensure that converges to a probability 
density. 

Definition 7 (Tightness). A family {ua,a G A} of probability densities in- 
dexed by A is tight if and only if for each e > 0, there exists an M such that 
f-M ^a(a:) da; > 1 — e for all a G A. 

Theorem 2. Suppose V (x) is chosen such that exists for <5 > 0 small enough 
and the resulting {u^ , d > 0} zs tight, then dx < oo and 

e~'^v{x) 

u^(a:)^u°(x)^j^—^^^ asS^O, 
where the convergence is pointwise. 

Shown in Figure 6 are the plots of for different <5 when V (x) = (a;^ + 
20(a; — 5) + c(a; — 5)^)/100 and c = 275. Here we choose S = 40/ N, i.e. [—20, 20] 
is discretized into N subintervals. Notice that the convergence speed is fast: even 
if the discretization is coarse, the resulting is still close to the final limit. In 
Figure 6, the two local minimums are at roughly the same level. By changing 
the value of c slightly, we can make one valley slightly deeper than the other. 
However, due to the exponential inverse relation of to V, this small change 
will be considerably amplified in u^. 
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Fig. 6. Left: V{x)] Right: for different 5 = ^ 



It is expected that the limiting distribution vP in Theorem 2 will also be 
the stationary distribution of the original stochastic differential equation: dXt = 
—XV {Xt)dt + dBt in the sense that if X(0) is distributed as vP independently 
of {Bt], then for any t > 0, the solution process Xt has the same distribution. 
We illustrated this in the following example. 

Example 1. (Ornstein-Uhlenbeck process) Solution Xt to the SDE dXt = 
fiXt + adBt is called the Ornstein-Uhlenbeck precess ([7]). Consider the case 
when (T = 1, /i = —a for some o > 0. Then by Ito formula, Xt = Xoe~°‘^ + 
/p If Xq is Gaussian 7V(0,cr) independently of {Bt}, then for each 

t > 0, Xt is also Gaussian with mean 0 and variance ^ -I- Let 

cr^ = then we can see that Xt has stationary distribution A^(0, with 
density function predicted by Theorem 2. 

Next we will discuss the limit behavior of first exit distribution of MC {Qn} 
from an interval. Consider MC {Q^} obtained in subsection 4.2. 

Lemma 9. Suppose ii < io < are nonnegative integers. Then the probability 
that starting from io hits first than it hits ii is: 



io — 1 

E 



n - 

J- J- n • 



^ 2-1 






Pj, 



n - 

J- X ri . 






Pj 



(15) 



Suppose a,b,c G K and a < b < c. For each (5 > 0, define if = [a/ 5], 
il = [b/5], = [c/5]. Then for the corresponding embedded MC {Qn}, the 

probability PtslTts < T^s) can be calculated by Lemma 9. The next theorem 
characterize the limiting behavior of such probability when i5 ^ 0. 



Theorem 3. Using the same notation as in the above paragraph. Then as 5 ^ 0, 



Pit {Pij. < Pij, ) 



/a ® dx 

U g-2V(x) 

J a 
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It can be shown that the above asymptotic expression coincides with the 
corresponding probability of the original diffusion process (see [4]). Furthermore, 
under some proper assumptions, the expected escape time from an interval of 
the embedded MC can be studied as well and can be shown to converge to 
the corresponding value of the original diffusion process. Therefore the DSHS 
presents a powerful tool for studying the sample path properties of the SDE, at 
least when the discretization step is small enough. 

The advantage of having closed form formulae for various properties of the 
stochastic hybrid systems is that it can greatly facilitate the design and evalua- 
tion of such systems. These topics will be pursued in future work. 
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Abstract. We address the problem of designing a distributed, hybrid 
factory given a description of an assembly process and a palette of con- 
trollers for basic assembly operations. In particular, we present a method 
that, starting with a product assembly graph (PAG), allows us to “com- 
pile” a factory description, consisting of a geometry and a hybrid, dy- 
namical system representing the motions of robots on that geometry. 
This method is based on a formalism, which we have described in previ- 
ous work, that allows us to manage the details of low level, continuous 
control of robot actuation and high level, logical control of various cou- 
plings of robot behaviors. The factory description is intended to be an 
aid in the design of an actual factory, if not directly implementable itself. 



1 Introduction 

Large distributed networks of robots and computers form the basis of mod- 
ern manufacturing systems. These systems should be rapidly reconfigurable, to 
adjust to design changes in the products they assemble or to changes in the mar- 
ket. Furthermore, they must be easily programmable. These goals, however, are 
seldom achieved in practice because of the complexity that hundreds of inter- 
connected, concurrently operating robots necessarily incurs. The programming 
process can be ad hoc and frequently results in a large fraction of the control 
code being “exception handler code” . This cost is felt in terms of expensive pro- 
gramming projects, incompletely understood factory behavior, and a delay in 
the introduction of new products to the market. 

In previous work, [11], [10], we described a formalism for representing and 
composing concurrent robotic systems which we believe addresses some of the 
problems in designing distributed, dynamic factories. Specifically, we introduced 
the notion of a Threaded Petri Net (TPN), which combines low level motion con- 
trol of individual robots or small groups of robots with high level logic control to 
manage how couplings between robots change over time in the factory. We also 
introduced a way of composing TPNs to create larger TPNs and demonstrated 
several properties of TPNs and our composition rules. Although we believe these 
tools will prove applicable to a broad range of automation settings, our notion of 
assembly is more immediately inspired by the high flexibility, low volume setting 
targeted by the “Minifactory” of Rizzi et al. [17], wherein decentralized general 
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purpose robotic agents accomplish all the factory’s parts transport and assem- 
bly operations in fluidly choreographed transactions. For example, a complex 
subassembly task requiring four or six coordinated degrees of freedom can only 
transpire in such a Minifactory when some subgroup of the decentralized robots 
“agrees” to collaborate closely in forming the specialized “machine” (the higher 
degree of freedom coordinated mechanism) suited to the specific task at hand. Of 
course, that alliance must be temporary, since each of the participating agents 
is required to play analogous but different roles in other machines, both prior 
and subsequent to the instantiation of the one in question. The TPN formalism 
provides tools to frame this problem. 

In the present paper, we apply our work on TPNs and composition to the task 
of automatically compiling factory descriptions from a standard representation of 
a product assembly process called a product assembly graph or PAG. The factory 
description that results consists of: an allocation of robots of various types; a 
geometrical description of the space that these robots inhabit; and a concurrent 
hybrid dynamic system, represented by a TPN, which directly corresponds to 
the robot programs. We use results from our previous work to show that the 
resulting TPN is live and that it successfully implements the process specified 
by the PAG input. 

It must be stressed that we presuppose an infrastructure of tunable and 
switchable feedback controllers which our compiler merely “puts together”, in 
a safe and correct way, to realize the assembly process. Such a palette of con- 
trollers is relatively easy to build for environments well described by generalized 
damper dynamics [12], but becomes quite challenging when dynamical dexterity 
is required. For example, in [2], substantial “hand building” affords deployments 
of controllers whose domains of attraction explicitly include portions of the for- 
ward limit sets of their neighbors. Here, we simply assume that these “dynamical 
systems details” have been worked out via parameterized families of regulators, 
and represented in a way that allows us to use them with TPNs (see Section 
4) . We then focus on the logical coordination and scheduling problems that fol- 
low. We have, in fact, built such a palette and a compiler for a simple class of 
PAGs and simulated the resulting factories. Animations of these factories can 
be viewed at http://www.eecs.umich.edu/~klavins/mf/. 

The paper is organized as follows. In Section 2, we review related research. 
In Section 3, we review TPNs and our composition method. In Section 4, we 
introduce mathematical models which represent robots, operations (those in the 
palette of controllers), and factories. In Section 5, we describe the compilation 
algorithm in detail and prove that it describes live and correct factories. Finally, 
in Section 6, we discuss a simple implementation of the compiler. 



2 Background and Related Work 

The research we report on in this paper draws from several areas: preimage 
backchaining of motion controllers, autonomous robot assembly, and hybrid dis- 
crete/continuous systems including Petri Nets. We review each of these areas as 
they pertain to the present research. 

Preimage backchaining was introduced into the motion planning literature in 
[14] as a method of sequentially composing motion strategies. In [2] this method 
was extended to dynamically dexterous robot manipulators in work that serves 
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as the basis of our current research. In [11], we expanded these ideas to include 
the notion of concurrent composition of behaviors for the case of several robots 
in a shared workspace based on simple Petri Net composition methods. Similar 
methods are found in work on the bottom-up synthesis of Petri Nets, especially 
[13], where simple Petri Nets are combined along paths and invariants of the 
resulting net are obtained from the constituent nets. In the present paper, we 
use the properties of our compositional tools design and verify an algorithm that 
automatically compiles concurrent, hybrid factories. 

The approach to assembly in [12], for simple situations, introduces an au- 
tomatic method for constructing a control law that guides a single robot to 
assemble a product from its parts based on the notion of an artificial energy 
landscape wherein the configuration of least energy is the one in which the 
product is assembled. It is not obvious that this method could be extended to 
three dimensional systems with orientable parts. In this paper we take the view 
that the PAG of a product corresponds to a discrete and parallelized version of 
such a potential function. The individual steps of the assembly may be given by 
artificial potential field controllers, but the overall logic of the assembly is given 
by the PAG. This allows us to use multiple robots, as in a high volume factory 
setting. 

Programs such as Archimedes [9] exist which transform the GAD description 
of a product into a PAG. Little research has been reported concerning translating 
the PAG directly into a layout and distributed program for a factory, although 
in the one example we know of, [19], the authors produce elementary conveyer 
belt layouts. In this paper we introduce a method that we believe will lead to a 
general procedure for carrying out such a translation. 

Hybrid systems combine a discrete state and a continuous state into the same 
model. A common representation is the hybrid automaton, [7]. Many definitions 
of hybridized Petri Nets, serving various needs, have also been investigated: Gon- 
tinuous and Hybrid Petri Nets [4] , Differential Petri Nets [5] , and DAE-Petri Nets 
[1]. The last is most easily seen as an extension of hybrid automata. Our defini- 
tion of Threaded Petri Net differs from these definitions in several regards. First, 
we consider a place in a net to be a controlled dynamic system on some subset 
of the degrees of freedom of the system, depending on the marking, and a transi- 
tion fires when and only when the systems in its preset are in stable equilibrium 
states. Furthermore, transition firings redistribute the degrees of freedom of the 
system to other dynamic systems in a controlled manner. 



3 Definitions and Basic Properties 

In this section we introduce the formal ideas that underlie our compiler research. 
We refer the reader to [10] for the details. We adopt the following definition of 
a Petri Net, also called a condition/event net, found in [8]. 

Definition 3.1 A Petri Net is a pair (T,P) where T is a finite set of elements 
called transitions and P C 2^ x 2^ whose elements are called places. 

We use standard Petri Net notation. If {{ai, ...,ai},{bi, ...,bj}} G P, we 
write [ai, ..., ap, bi , ..., bj] G P. If p = [oi, ..., ap b \, ..., bj] then left{p) is the set 
{a\,...,ai} and right{p) is the set {bi,...,bj}. A marking of a net (T,P) is a 
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set m C P. The flow relation F of a Petri Net (T, P) is the relation where 
(t,p) G F if t G left{p) and (p,t) G F if t G right{p). The preset of an element 
a; G T U F is set {y I y F a;} and is denoted *x. The postset of x is the set 
{y \ x F y} and is denoted x* . See [15] for a detailed introduction. 

In a graphical representation of a Petri Net, places are represented by cir- 
cles and transitions by squares. In our research, a place represents a controlled 
dynamical subsystem decoupled from the entire system in question. Transitions 
represent discrete changes in the dynamics of subsystems. 



3.1 Threaded Petri Nets 

Suppose we have a collection of robots ri, ..., r„ with configuration spaces C(ri), 
..., C(r„) whose continuous state can be given by x = (xi, ...,Xn) G C(ri) x ... x 
C(rn) and whose global dynamics is simply x = u. The dynamics of components 
of X are almost independent of each other. However, robots do interact for short 
periods of time, as for example during a parts mating operation, so that the 
dynamics of certain components of x may occasionally be tightly coupled. 

To describe how couplings change and which dynamics are operating on 
which components of x, we introduce the Threaded Petri Net, or TPN. Places 
correspond to control modes which we will have chosen from a palette of such 
modes. Thus, for each place p there is a system given by y = Fp(y) where y is 
the vector concatenation of Ip vectors (components of x) and Fp is chosen from 
the palette of controllers that we assume is already constructed. The mode has 
domain of attraction T>p and goal set Qp. Formally, 

Deflnition 3.2 A Threaded Petri Net (TPN) eonsists of 

1. a set T of transitions; 

2. a set P C 2^ X 2^ of places; 

3. for each p G P, size, dynamics, domain and goal Ip, Fp, Vp and Qp; 

4-. for each e € T a bijective function 

de ■■ U M X {!,..., ^p} ^ U ^ 

pG *e qGe* 



called the redistribution function of e; 

subject to the condition that for each e G T, 

pG *e qGe* 



(so that it is possible for de to be bijective). 

Note that the difference between a TPN and a condition/event net is not only 
the additional information associated with each place. We have also added the 
redistribution functions, de for each e G F, which define what happens to each 
component of x as mode changes occur. Graphically, a TPN is depicted as is a 
simple Petri Net, except that the redistribution functions are shown by curves 
through the net. See Figure 1 for example. 
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drop 



waitj 




pick2 



Fig. 1. An example of a Threaded Petri Net which describes the dynamics of 
three robots in a parts mating procedure. 



Definition 3.3 A marking is a pair {m, fm) where m C P and 

fm- y M X ^ 

p^m 



which specifies which degrees of freedom of the system each mode is operating on. 
A legal marking is one where fm is bijective. We will be concerned only with 
legal markings in what follows. 

A legal marking (m, fm) of a TPN says, for each p G m, which components of 
X Fp is acting on and what the dynamics of each component of x are. Thus, 
we can say how the state of the system is changing given a particular marking 
(m, /m). Given f G N, suppose that fff{j) = (pf). That is, under the marking 
(to, fm) the jth component of x is changing according to the Ah component of 
the mode dynamics of p: 

Xj = TTi o Fp{xf^(^p i ^, ..., 

where gives the Ah component of the function Fp. This is valid until some 
mode changes, which leads us to a definition of how events are triggered. 

Definition 3.4 Let (m, fm) be a legal marking, e G T is w-enabled with re- 
spect to X G R" if 

1. *e C TO and e* Dm = 0; 

2. for eachpG *e, (a:/„(p,i), ..., G Gp; 

3. for each q G e*, (x ^ Uodp(q,i)) ^ ^ 9 - 
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Notice that condition (1) is just the usual definition of m-enabled for condi- 
tion/event nets. The second two conditions impose the restriction that the dy- 
namic systems in the preset of the enabled event must be in goal states and the 
systems in the postset must all be prepared. 

A set of events G C E is called detached if whenever Ci and 62 are distinct 
events in G, *ei D *62 = ei* nc2* = 0. Suppose we have a marking (m, fm)- The 
follower marking (m', fm') with respect to G C if is calculated as follows. As 
with condition/event nets m' = (m — *G) U G*. fm' is the function given by 



fm'{p,j) 



fm{p,j) if p€m - *G 1 
fmO df'^ipG) otherwise j 



where e is the single event in p* n G. We write (/m,w) {fm','m') when 

ifm', Tn') is the follower marking of (/, m) with respect to G. Since legal markings 
i'rn, fm) are such that fm is bijective, we can be sure that every component of 
X is accounted for when the system is in the set of modes given by m. It can 
be shown that if (fm,Tn) is a legal marking and if (fmjiTi) {fm'i’ni'), then 
ifrn'j'm') is a legal marking as well. 



3.2 Composing Threaded Petri Nets 

As mentioned, we intend to compose TPNs into factories. We present a simple 
type of composition to complete this section. It is based on the idea of a cyclic 
subprocess, which we call a gear, and which we use as the basic building block 
of our nets. A gear represents the simplest thing a robot in a factory can do, 
besides remain idle: cycle repeatedly through some set of behaviors. 

Definition 3.5 A k-gear is a net (T,P) where T = {to, .--Ak-i} and P = 
{[ti] ti+i] \ i G Z/ k} . m C P is a legal marking for a k-gear if |m| = 1. 

(We ignore the dynamics and redistribution functions for now.) A gear for a 
robot models the program of a single robot. Certain places of a gear must be 
synchronized with the gears of other robots. Thus, we compose gears as follows. 

Definition 3.6 A gear net is defined recursively: 

1. A gear is a gear net. 

2. If {T, P) is a gear net and {S, Q) is a gear then (T U S', P U Q) is a gear net 
as long as the following conditions hold: 

(a) let {Ti, Pi), (Tk, Pk) be the set of gears in (TUS, PUQ) which intersect 
(S, Q). Then n^=i Pi = {[o; ^]} and fj^i Pi = {a, b} for some transitions 
a and b; 

(b) there exists a transition c G S — T such that [c; a] G Q. 

A legal marking for a gear net is one in which each gear in the net is marked 
exactly once. 

Since all places in a gear net are of the form [x; y], gear nets are a kind of marked 
graph, a class of nets which have been extensively studied. (See [3], for example.) 
Conditions (a) and (b) require that gears be added with a “standard interface”. 
We can show the following properties about gear nets. 
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Theorem 3.1 (Liveness) Gear nets are deadlock free under legal markings. 

Theorem 3.2 (Reversibility) Gear nets are reversible given any legal initial 
marking. 

Thus we are assured that systems we build up from gear nets are live, logically 
conflict free, and cyclic processes. 

4 Representation 

Next we describe how to represent the building blocks of factories - products, 
robots, workspaces and controllers - in a way that is amenable to compilation. 



4.1 The Product Assembly Graph 

A product assembly graph or PAG, is represented as a tree whose leaves 
represent parts and whose internal nodes represent operations on subtrees which 
yield subassemblies. For a given set of operations and part types we can define a 
simple class of PAGs as follows. Suppose that we have part types parti, ...,partk 
and operations 0\, ...,Oj where Oi is an operation which takes mi subassemblies 
and produces a single subassembly. Then the class of PAGs is given by: 

1. parti, ...,partk are all PAGs; 

2. for each i G {l,...,j}, if Pi,...,Pmj are all PAGs then Oi{Pi, ..., Pm,) are 
PAGs as well. 

Glearly, this defines a very simplified class of PAGs. In practice, each operation 
can take only certain types of subtrees (those representing subassemblies ap- 
propriate to the operation), operations are parameterized, and so on. However, 
we believe that this is a first approximation to the kind of PAGs that we will 
encounter in practice. 

For a given PAG P, we give a unique label to each node P' in P, called 
Label(P'). This identifies the subassembly that is result of the operation. 



4.2 Robot Types and Workspaces 

We suppose that there is some set of robot types at our disposal which we denote 
by T = {Ti, T 2 , ...}. Each type T has an “ideal” workspace W{T) C (compact 
and connected) and a conflguration space C{T). W{T) describes the geometry 
of the set of all positions the robot may take - in general, a solid in C(T) 
represents the degrees of freedom of the robot. An example robot type in the 
Minifactory is the courier, a two degree of freedom planar robot with a workspace 
that is a rectangular solid [x^m, Xmax] x [ymm, y-max] x [0, /i] where the x and y 
terms represent the limits of movement on a factory platen and h is the height 
of the robot. The conflguration space of a courier is just 

An instantiation of a robot will be denoted by an identifier r with type 
Type{r) G T, workspace W’(r) ~ W(Type{r)), and conflguration space C(r) = 
C{Type{r)). As we build factories in the compilation procedure defined below, we 
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instantiate new robots and add them to a set R of robot identifiers. We suppose 
that their ideal workspaces are copies of the ideal workspaces of their types and 
that for any two distinct instantiated robots ri and r^, we have >V(ri) n >V(r 2 ) = 

0. We represent the way in which robots are located with respect to each other 
by forming an identification (quotient) topology on the union of the workspaces 
of the robots. This does not represent the actual layout of the factory because 
the resulting geometry may not embed in without some “stretching”. We 
comment on the layout procedure in Section 5. 

A Robot can carry a subassembly, which may be an atomic part or the result 
of some operation on some number of parts. Which subassembly, if any, a robot 
is carrying is the discrete state of the robot. It is given by Label{P) for some 
node P of the PAG that is being compiled. The distinguished label nopart will 
be used to denote the state of a robot not carrying any subassembly. 

4.3 Templates for Controllers 

In order to use controllers with our assembly compiler, they must be represented 
in a standard way. Here we describe a template for representing controllers. This 
template consists of: a description of the robots needed; the index of the robot, 
called the “carrier”, that will hold the result of the operation once the it is 
complete; a way of combining the workspaces of the robots into a workspace for 
the operation; a parts transform pair, and a control law over the configuration 
space. The carrier robot and its workspace are used to join the workspaces of 
controllers as the PAG is traversed during compilation. We have the following 
definition: 

Definition 4. 1 An operation template is a tuple 

0=(R,j,~,(a; b),F) 



where 

1. R = (Ti, ...,Tfe) is an ordered set of types of robots, with k = |i?|; 

2. j G {1, ..., k} is the index of the robot that will carry the result; 

3. ~ is an equivalence relation. W = (Ui=i ^^6 resulting workspace; 

4- (a; b) = (oi, ..., Ok] b \, ..., bk) is the parts transform pair denoting how the 

labels of the parts each robot is carrying change as a result of the controller 
reaching its equilibrium state; 

5. F is a vector field on C describing the controlled dynamic system corre- 
sponding to the controller with domain Vp and goal Qp. C is defined by 
Y\p^PiC{T) — A where A is the set of configurations which correspond to two 
robots touching or being in the same place according to ~. 

The operations of interest take some number of subassemblies and perform an 
operation that produces one new subassembly. Thus, bj yf nopart while bi = 
nopart for i ^ j. 

An instantiation of a template is an assignment of robot identifiers to R 
and is written 0{r\, ...,rk). The Threaded Petri Net fragment corresponding to 
this instantiation is denoted No{r\, ...,rk) and is depicted, in its general form, 
in Figure 2. 
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wait(ri,a-\) 



wait(r2,a2) 



wait(ri^,aiJ 




wait(r\,b-\) 



wait(r2, b2) 



wait(r^,ht) 



Fig. 2. The Threaded Petri Net associated with the instantiation 0(ri, r^) 



4.4 Factories 

We define a factory to be a workspace, a set of robots in the workspace, and a 
TPN describing the dynamics of the factory. This structure will be built up as 
the compilation procedure progresses. It will start with a single robot whose task 
it is to receive the final subassembly from the highest operation in the PAG. 

A factory, therefore, is a triple T = {R,^,N) where i? is a set of robot 
identifiers, ~ is an equivalence relationship on the union of the workspaces of 
the robots which describes how the robots are placed in the factory, and N is a 
TPN which describes the hybrid dynamics of the factory. 

5 The Compilation Algorithm 

In this section we describe the general form of the compilation procedure for a 
given class of PAGs. We assume that each operation is already described via a 
template and that templates for the operations for picking up parts (from parts 
feeders or trays) and dropping off the final subassembly part are also given. As- 
sume that the type of robot that receives the final subassembly is OuputType. 
The input to the algorithm is a PAG P = 0{P\, ...,Pk)- The function Gompile 
initializes the factory structure with a robot and workspace for the final sub- 
assembly DropOff operation and then calls the main function GompileNode. 

Gompile (P) 

r <— Instantiate{OutputType) 

R ^ {r} 

^ {(x,x) \ xGW{r)} 

N ^ No 
GompileNode(P, r) 

End 

Here, N is initialized to Nq which is the Fragment depicted in Figure 3. 

The subroutine GompileNode first adds to the factory the robots and workspaces 
required for the operation O and then applies itself to each of the subtrees Pi 
through Pfe. Assume that P = C>i(Pi, ..., P^J whereO* = {Ri,j^, (a*; b*), P^) 

with Pj = (Pi, ...,Tfc) 
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wait(r, FinalAssem) DropOff(r) wait(r, nopart) 



Fig. 3. The Threaded Petri Net Nq used to initialize the factory before compi- 
lation. FinalAssem is the label of the root of the input PAG. 



CompileNode ( P, carrier ) 

Allocate robot identifiers r/ where Type{r{) = Ti for each I ^ j 
rj <— carrier 
R ^ RU {ri,...,rk} 

HUH) 

N ^ NU Noi{ri,...,rk) 

For each I G {1, ..., k} 

If ai yf nopart Then 

Choose P G {Pi, ...,Pjni} such that Label{P) = at 
CompileNode(P, r/) 

Endlf 

EndFor 

End 



The CompileNode routine first allocates the new robot identifiers needed for the 
operation. The factory robots are updated to include these robots as well as 
the carrier. The equivalence relation is updated as well and becomes a relation 
over the union of workspaces of all the newly allocated robots as well as the 
robots that were already in R. Then the TPN that describes the dynamics of 
the factory is updated to include the fragment for the operation. Finally, for 
each robot identifier r which, according to the part transition pair (a; b), should 
be arriving at the current operation with a part o, CompileNode calls itself on 
the subtree corresponding to a with r as the new carrier robot. Notice that the 
recursion eventually bottoms out since the part nodes (leaves) of the PAG have 
no children. 



5.1 Properties of the Resulting Factory 

We can show that the factory is a gear net and that its dynamics are correct. 
We first make use of the following lemma. 

Lemma 5.1 Let (Ti, Pi), (Tk, Pk) be gear nets and suppose that for each 
i G {l,...,fc} we have that ({ 0 ^, 6 ^}, {[a : 6 ]}) C (Ti, Pi) is the intersection of 
some number of gears in (Ti,Pi). Then, the net obtained by identifying each 
{{ai,bi}, {[a : 6 ]}) is also a gear net. 

This result can be used to show that the algorithm above produces gear 
nets. The proof is inductive on the form of the PAG input. Roughly, we show 
that PAGs consisting of a single part produce single gears and that assuming 
that the algorithm compiles gear nets for the subtrees Pi,...,Pk of the tree 
P — 0{Pi, ..., Pk), we show that it compiles P correctly into a gear net as well. 
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Theorem 5. 1 Let N be the TPN resulting from applying the above algorithm 
to the PAG P. Then N is a gear net. 

Since gear nets are live and reversible and because they are deterministic it is 
also straightforward to show that under any legal initial conditions (we usually 
consider the situation where each robot is running wait(ri, nopart) as the initial 
marking), that the output robot runs the controller for the DropOff operation 
infinitely many times in any run. Formally, 



Theorem 5.2 Suppose that mo,mi, ... is a sequence of markings obtained from 
a run of the gear net N produced from PAG P. Then there exist infinitely many 
markings m in the sequence such that DropOff{r) G m where the robot r is the 
one instantiated in the initialization routine Compile. 

The workspace that results from compiling a PAG, W = 
does not represent the layout of the factory. In general, W needs to be “stretched” 
to be properly embedded in if it is even possible to do so. At present, we 
do not have a complete procedure for producing this layout, however, we have 
an idea of how it will be carried out in practice. Certain workspace types are 
amenable to stretching in certain directions. For example, the workspace of a 
planar robot may be extended to be longer or wider but not taller. Thus, there 
is an allowable family of embeddings T from W into which must be explored. 
Once one is found, say f G !F, the controllers for the low level operations are 
composed with / to produce dynamics on the image of /. In the next section, 
we illustrate this procedure in a simple implementation. 



6 The DotFactory: An Example 

We have explored the compilation procedure with a simple family of PAGs and 
a class of “toy” factories called “DotFactories” . In the simplest of our investiga- 
tions, we assume that there is only one part type, atomici), and two operations 
mate{-,-), weld{-). The robots we consider are all of the same type Tdot with 
workspaces that are copies of the unit interval [0, 1] G K and configuration spaces 
[0, 1] (guidepaths) . The physics are simplified: a robot may control its velocity 
directly (i = u); parts move with the robots nearest to them; and part transfers 
happen instantaneously as long as the robots involved are close together. Robots 
have width r. 

An example template is given next, for the mate operation, mate = (i?, j, 

(a; b), F) where 

1. R = {Tdot, Tdot, Tdot)', 

2. j = 3; 

3. ~ = {Ai = A 2 = R 3 } where we assume that robot i will have as its 
workspace the interval [At, Bt] G M; 

4. (a; b) = {LABi^ LAB 2 , nopart; nopart^ nopart, LABf); 

5. F is a control law over Wmate = (Ui=i[^»> ^»D/ ~ ^ = W^ate and 

Q = Re(Ai -I- 2r, A2 + 2r, B3) (a small open ball around the goal point). 
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We omit a description of the details of F. In the actual implementation, F is 
derived from a navigation function [16] - a method that is quite suitable to the 
present situation. Similar templates are given for the weld, atomic and dropoff 
operations. 

The input PAG is represented syntactically as in the following example input 

file: 

6 parts; // the number of subassemblies 
root = sub3; // the finished product 

sub3 = mate ( sub2, parts ); // how to make the subassemblies 

sub2 = weld ( subl ) ; 

subl = mate ( parti, part2 ); 

parti = atomicO; // these are the actual parts 
part2 = atomicO; 
parts = atomicO 

The TPN that is compiled from this PAG describes programs and low level 
control for six robots in a workspace composed of guidepaths. Since the PAG is 
a tree, the compiler constructs workspaces that are, topologically, trees as well 
so that the layout procedure is obvious. The programs for each robot, essentially 
gears, can be read off directly from this TPN. For example, the gear for the 
robot, call it r^, that receives the result of subassembly 1, fixes it to be welded, 
and then mates it with part 3 is 

Loop: 

If states = nopart 

Run is = wait Until states = parti A states = part2 
Run is = 7T3 o mate{xs, xg, xf) Until xs = subl 
Break 

If states = subl 

Run is = hold Until statei = nopart 

Run is = 7Ti o weld{xs, X 4 ) Until states = sub2 

Break 

If states = sub2 

Run is = wait Until states = nopart A states = parti 
Run is = 7Ti o mate{xs,X 2 ,xi) Until states = nopart 
Break 
End Loop 

Programs for the other robots are similar. Note that we assume a simple com- 
munication system which, in our implementation, is composed of two parts: a 
shared memory where robot i may write its discrete state (the label of the part it 
is carrying) to memory location i and may read any memory location; and a high 
speed continuous state sharing link between robots sharing control modes. Be- 
cause of the distributed nature of the control, the number of continuous states a 
robot must monitor at any time is less than or equal to the size of the the largest 
control mode, independent of the size of the PAG and the resulting factory. We 
believe that the method will scale well to significantly larger factories. 

Each robot is simulated concurrently at varying operating speeds (chosen 
randomly) and with varying control speeds. All factories that were compiled 
performed well under these minor disturbances due to the reactive nature of the 
low level control method used (borrowed from [16]) and to the robust nature 
afforded by the gear net structure of the compiled TPN. 
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We have also investigated robot types with workspaces that are “T-shaped” 
and shared by another robot. We use the method suggested by Christ and 
Koditschek in [6] for constructing dynamical systems of multiple points on topo- 
logical graphs. Animations of the factories resulting from several different input 
PAGs can be viewed at http://www.eecs.umich.edu/~klavins/mf/. 

7 Conclusion 

We have developed an automatic factory compiler based on our formalism for 
representing concurrent, hybrid systems. The compiler uses a standard represen- 
tation of robot workspaces and low level operations and yields a factory geom- 
etry, robot task allocation and control programs for each robot. The resulting 
factory dynamics are shown to be correct using basic properties of our gear net 
composition method. Our implementation of a simple toy situation suggests that 
our method yields robust systems and that it scales well. 

In the future, we will consider optimizing the compiled net for robot reuse (i.e. 
reallocating tasks so that one robot alternates between tasks formerly assigned 
to two robots) and for parallelization of tasks. This leads to TPNs that are not 
based on gear nets but do have a regular structure, and implies the need for a 
much more sophisticated layout procedure. The dynamics of the resulting nets 
must be considered with fairness constraints so that they do not deadlock. We 
must also address the issues of error recovery and product reworking. We believe 
that the complexities these issues introduce into our TPNs can be managed by 
compositional methods similar to those we have already introduced. We are also 
working on applying these ideas to a factory design tool for a more realistic 
example which better approaches the Minifactory, mentioned in Section 1 [17]. 

This research has also lead us is to study the idea of “momentum across 
transitions” where the dynamical systems corresponding to places are not always 
controlled to equilibrium states. For example, a robot might toss a ball to another 
robot which must catch the ball. As the ball approaches the second robot, the 
transition of that robot into a catching behavior becomes more urgent. We would 
like to be able to solve this problem not with the explicit use of time as in timed 
Petri Nets but rather with the intrinsic dynamics of, in this case, a ball in flight. 
An example of switching between tasks based on urgency can be found in [18] 
where Rizzi controls a robot to switch between the tasks of bouncing one of 
two balls on a paddle, effectively juggling them. A systematic approach to this 
problem may yield factories that are highly dexterous, distributed manipulation 
systems. 
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Abstract. In this paper, we demonstrate a novel hybrid control synthe- 
sis approach using an automotive suspension system. Discrete abstrac- 
tions are used to approximate the continuous dynamics and emphasis is 
placed on the nondeterministic nature of the abstracting models. The 
regulator problem for hybrid systems is formulated for safety specifica- 
tions and algorithms for control design are presented. 



1 Introduction 

In this paper, a novel systematic methodology for hybrid control synthesis is 
presented and an example of an automotive suspension system is used to illus- 
trate the approach. The main advantage of the approach is that it provides a 
convenient general framework for hybrid systems not only for analysis, but more 
importantly for controller synthesis. Discrete abstractions of the continuous dy- 
namics are studied and the emphasis is placed on the nondeterministic nature of 
the abstracting models. The notion of quasideterminism is used to characterize 
discrete abstractions that can be used for control design. The class of systems 
we are particularly interested in is the class of piecewise- linear systems. Note 
that the analysis and synthesis algorithms have been implemented using general 
purpose software, namely Matlab, Simulink, and Stateflow. 

Early results of the approach have appeared in [7,6]. The approach has been 
influenced particularly by [1] where a feedback architecture of a continuous plant 
with a discrete-event controller is used for hybrid control design. Piecewise-linear 
systems evolving in discrete-time have been studied in [11,13] and they represent 
an important class of systems with many practical applications. Recently, the 
class of piecewise-linear systems has attracted the attention of many researchers, 
see for example [5,2]. Analysis and synthesis methodologies based on discrete 
abstractions have been studied extensively in the hybrid system literature [9,8]. 

The paper is organized as follows. The automotive suspension system is in- 
troduced in Section 2. In Section 3, the modeling formalism is briefly outlined. In 
Section 4, the deterministic nature of the discrete abstractions is discussed and 
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algorithms for the computation of the discrete approximations are presented. 
Finally, the regulator problem for hybrid systems is formulated in Section 5. 

2 Automotive Suspension System 

This example describes a simplified model of an automotive suspension for an 
independent wheel. The diagram of Figure 1 illustrates the modeled character- 
istics. We represent the suspension as a spring/damper system equipped with 
a compressor and an escape valve. We concentrate only on bounce degrees of 
freedom, which are represented in the model by the vertical displacement and 
velocity. The chassis level is raised by pumping air into the system and lowered 
by opening an escape valve. The suspension influences the bounce according to 
the equations 



F = —2k{z + h) — 2cz (1) 

mz = F — mg + u (2) 

where z, z, and z are the vertical displacement, velocity, and acceleration re- 
spectively. The spring and damping rate of the system are represented by the 
constants k and c. There are two inputs to the model. The first input is the road 
height h caused by irregularities in the road surface and the second input is the 
force u caused by the air pressure of the compressor or the escape valve. 




Fig. 1. Automatic height control system 



The principal objective in this example is to design an automatic height 
control system, which increases driving comfort, allows the driver to select the 
chassis level according to off-road and on-road conditions, and does not violate 
driving safety. We consider two driving modes for the system, straight and curve. 
While in straight driving mode, the driver or a higher level control system in an 
autonomous vehicle, selects the set-point {sp) for the vertical displacement. The 
objective of the controller is to guarantee that the vertical displacement remains 
in a tolerance interval [sp — lt^ sp+ht\ for any road disturbance from a prescribed 
bounded set. While in curve mode, the requirement is that the control system 
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does not influence the chassis level, using either the compressor or the escape 
valve, so not to violate the safety of the system. 

In this paper, the design of the controller that selects the action of the com- 
pressor and the escape valve is formulated as a hybrid control synthesis prob- 
lem. A controller is designed based on discrete abstractions of the continuous 
dynamics using the refinement algorithm presented in Section 5. The controller 
is responsible for generating the control laws that guarantee that the chassis 
level will track the set-point within the prescribed tolerance while in straight- 
driving mode and will suspend the active control while in turning mode. Note 
that pneumatic suspension system examples have been used in the hybrid sys- 
tem literature to illustrate verification algorithms in a linear hybrid automata 
setting [4,14,3]. 

3 Modeling of Hybrid Systems 

3.1 Hybrid System Model 

We propose to model hybrid systems as set-dynamical systems [10]. A set- 
dynamical system (SDS) is denoted as {X, U,Y; f, g) where X is the state set of 
the system, U is the input set, Y is the output set, f : X x U ^ X is the state 
transition function, and g : X xU ^ T is the output function. It is important to 
distinguish between the controlled and the uncontrolled inputs (disturbances) of 
an SDS. Furthermore, in the case when the measurements are different than the 
outputs, a measurement set M and a measurement function m can be included 
in the system’s description. 

In order to describe the behavior of a dynamical system, the notion of time 
must be included in the system’s representation and this is accomplished with 
an index set J equipped with a simple order relation. Assume that the index 
set J is given. Define index functions a : N ^ J. An index function is said 
to be admissible if rii < ri 2 a{n\) < a(ri 2 ) (i.e. a is order preserving), and 
ni yf ri 2 a{ni) yf 0 (^ 2 ) (i.e. a is injective). The state a; G AT is associated 
with an index j(n) meaning the state at time j{n). 

A hybrid dynamical system (HDS) is defined as an SDS where the constituent 
sets consist of a continuous and a discrete part. We assume that the continuous 
part is a subset of a finite dimensional vector space and that the discrete part is 
finite. 

Definition 1. A hybrid dynamical system is defined by {X,U, D,Y, M; f, g,m) 
where X = Xc x Xd is the state set; U = Uc x Ud is the set of control inputs 
consisting in general of continuous and discrete controls; D = DcX Dd is the set 
of disturbances; Y = YcXYd is the output set; M = Me x Md is the measurement 
set; f: XxUxD^X is the state transition function; g: XxUxD^Y is 
the output function; and m : X x U x D ^ M is the measurement function. 

Presently, we have focused on piecewise-linear systems [11,13] to facilitate 
the development of analysis and synthesis tools. These systems arise when the 
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state set and/or the input set are partitioned into regions described by linear 
equalities and inequalities and the dynamics at each region are described by 
linear (or affine) state transitions. Output and measurement maps can be defined 
also in a similar way. The class of piecewise-linear systems is quite general as it 
includes linear systems, finite state machines, and their interconnections. They 
can be used also in many instances as approximations of more general systems. 

Control specifications and primary partition Control specifications for hybrid 
systems can include safety requirements that are usually formulated with respect 
to a partition of the state space of the system. Consider the state set X of an 
SDS and define the mapping tt ■. X ^ P(A1) from X into the power set of X . The 
mapping tt defines an equivalence relation on the set X in the natural way 
x\ Ett X 2 iff 7r(a;i) = tt(x 2 )- The image of the mapping tt is called the quotient 
space of X by and is denoted by Adopting this notation we can write 

7T : A — > X/Et^ where tt is understood as the projection of X onto XjE^^. The 
mapping tt generates a partition of the state set X into the equivalence classes 
of Et^ and will be called generator. We assume that the partition defined by 
7T is appropriate for extraction of important information for the system and it 
will be called the primary partition. More specifically, we are interested in the 
case when X = M” and the generator is defined by a set of hyperplanes in 
R". Note that such piecewise-linear regions arise in many applications. Consider 
the collection {/ii}i=i, 2 ,...,^j hi : M" ^ R of real-valued functions of the form 
hi{x) = gj x — Wi, i = 1,2, ... ,£ where gt G R” and Wi G R. Let Hi — ker{hi) = 
{a; G R” : hfix) = gf x — Wi = 0} and assume that Hi is an (n — l)-dimensional 
hyperplane {Vhfix) = gJ 0). We define the function : R” ^ {—1,0, 1} by 

( -1 if hi{x) < 0 

K{x) = ^ 0 if hi{x) = 0 (3) 

{ I if hi{x) > 0 

Then, the generator is defined by 7r(a;) = [hfix), . . . , fn{x)Y' . Although the gen- 
erator has been defined as tt : R" — > {—1,0,1}^ there is a bijection between 
{ — 1,0, 1}^ and the quotient set XjE^^ (they are the same set). 

Measurements and final partition Suppose that at time k we have that y{k) = 
TT{x{k)) G XjE^^. If it is agreed that the granularity of the partition generated by 
the mapping tt is appropriate for the extraction of useful information regarding 
the system’s behavior, then it is desirable to uniquely determine the state at the 
next iteration up to its membership on an equivalence class y(k -I- 1) = Ti{x{k + 
1)) G X/Et;.. This can be accomplished by considering a finer partition than the 
partition defined by the generator tt to obtain better estimates for the continuous 
state. This partition will be called the final partition and will be determined using 
the quasideterminism property discussed below. The generator irp is defined in 
a similar way as the output function tt. Given a partition defined by a finite set 
of (n — l)-dimensional hyperplanes the generator wp ■ X ^ XjE^rp separates 
the state space into a finite number of equivalence classes which correspond to 
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polyhedral regions. The function z = can be viewed as a measurement 

function that provides some information about the continuous state. Intuitively, 
our ability to make decisions to influence the behavior of the system depend on 
the amount of information contained in the measurement signal. 

Example - The automotive suspension system The system contains continuous 
dynamics due to the spring/damper subsystem and discrete dynamics due to 
pneumatic part of the suspension. Furthermore, the control specifications con- 
tain constraints for both the continuous and discrete variables. For these rea- 
sons, the automotive suspension system is modeled as the hybrid dynamical 
system {X, U, D,Y,M; /, g, m). The state space of the system is X = XcX Xd = 
X {straight, curve} representing the displacement and the velocity of the 
system, and the driving mode. The set of control actions is U = {uq,u\,U 2 \ 
corresponding to the case when the controller is suspended, the compressor is 
on, and the escape valve is open respectively (the compressor and the valve can 
not operate simultaneously). The set of exogenous input (that cannot be con- 
trolled) is D = Dc X Dd = K X {turn, resume} representing the road height 
and the selection for the driving mode respectively. The output set is T = M 
representing the chassis level. The measurement set is described as the quo- 
tient set XfE^p induced by the final partition irp that is to be determined in 
Section 4. The state transition function f: XxUxD^X is described by 
x{k + 1) = Ax{k) + Bu{k) + Ed{k) where x\ is the displacement of the chassis, 
X 2 is the velocity, u is the applied force due to either the compressor or the 
escape valve, and d is the road height. The parameters of the system A, B, and 
E are derived from the differential equations (2) by sampling at a prescribed 
rate T. Finally, the output function is y{k) = Cx{k) where C = [1,0] and the 
measurement function z{k) = TTF{x{k)) returns the membership of the state in 
one of the equivalences classes of the final partition. 

3.2 Control Specifications 

Regulatory feedback control of hybrid dynamical systems is based on a repre- 
sentation of the control specifications as a set-dynamical system which is usually 
called the exosystem. In this paper, we focus on the case when the exosystem 
is described by a finite automaton. The case when hard time constraints on the 
transitions of the exosystem are necessary can also be studied in this framework 
by including clocks in the description of the plant. 

Example - The automotive suspension system The control specifications for 
the automotive suspension system are now described. While in straight driv- 
ing mode, the driver or a higher level control system in an autonomous vehicle, 
selects the set point (sp) for the vertical displacement. The objective of the con- 
troller is to guarantee that the vertical displacement remains within a tolerance 
interval [sp — It, sp + ht] for any road disturbance from a prescribed bounded 
set. While in curve mode, the requirement is that the control system does not 
influence the chassis level, using either the compressor or the escape valve, so not 
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to violate the safety of the system. The control specifications can be described 
formally by the finite automaton shown in Figure 2(i). The state eg corresponds 
to the case the driving mode is straight, where the requirement for the chas- 
sis height is to be inside the tolerance interval [sp — It, sp + ht]. The states ei 
corresponds to the case when the driving mode is curve. The input alphabet is 
S = {turn, resume, e} where e is a void event. 

The primary partition can be derived from the control specifications in 
a straightforward manner and is described by h\{x) = x\ — {sp + ht) and 
h2{x) = x\ — {sp — It). Then the generator is defined by tt{x) = 
where the function hi is defined in Equation (3) and it separates the state space 
into five equivalence classes. For simplicity, we will consider that the safe re- 
gion is described by the closed interval [sp — It, sp + ht] and will consider only 
three regions corresponding to safe, high, and low chassis levels as shown in 
Figure 2(ii). 




Fig. 2. (i) Exosystem, (ii) Primary Partition 



The finite automaton of Figure 2(i) can be represented by the set-dynamical 
system {Xe,Ve,Ye, Mg', fe, ge,iTie) where Xg = {eo,ei} is the state set, Vg = 
{turn, resume, e} is the set of exogenous inputs, Yg = {turn, resume, e} is the 
output set (which characterizes part of the exogenous inputs to the plant), and 
Mg = XjET^ is the set of output requests. The state transition function fg : 
Xg X Vg -I- Xg is the state transition of the automaton, the output function 
ge : XgXVg ^ Yg is defined as gg{e, v) = v for every e € Xg and v € Vg. Finally, 
the output request (measurement) function is defined as follows. 



mg{e,v) 



safe for e = eg, \/v € Vg, 

y G {safe, low, high} for e = ei, Vu G V), 



(4) 



4 Partition Refinement and Discrete Abstractions 

4.1 Motivation 

In order to analyze hybrid systems and design control algorithms, it is desirable 
to induce dynamical systems in finite quotient spaces that preserve the properties 
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of interest and then study the simplified models. Let / be the state transition 
function of an SDS and assume that the inputs are fixed. Consider the diagram in 
Figure 3-(a). Intuitively, the map tt is used to coarsen the state set of the system. 
The question that arises is whether the system / can follow this abstraction. This 
question is concerned with the existence of a mapping / : X/E^^ — > X/if,r that 
makes the diagram commute. It is shown in [10] that / exists if and only if 

xiE^X2 ^ {tt o f){xi) = {n o f){x2) (5) 

(where o denotes function composition) and moreover, if (5) is satisfied then / 
is unique. Note that the above result does not require any structure on the set 
X or the mappings tt and /. Using equivalence relations on the state set A", it is 
possible to define new dynamical systems in the derived quotient spaces. These 
systems are called induced dynamical systems. 

4.2 Quasideterminism 

Quasideterminism can be viewed as a desirable property of the partition of the 
continuous state space. The central characteristic of quasideterministic systems 
is that only the reachability properties with respect to the control specifica- 
tions are preserved in the quotient system resulting in more efficient algorithms 
to partition the state space that are applicable to larger classes of hybrid sys- 
tems. Quasideterminism is a weaker requirement than the existence of a finite 
bisimulation. A partition that results in a quasideterminism can be always be 
computed for piecewise-linear systems, while recent results have shown that fi- 
nite bisimulations exist only for limited classes of systems [8] . In both approaches 
an algorithm is used to refine the state space. A bisimulation corresponds to a 
fixed point of the refinement algorithm. In quasideterminism, we do not require 
the existence of a fixed point but we stop the refinement at a prescribed fixed 
iteration. The disadvantage of that is that in this case the quotient system does 
not completely preserve the reachability properties of the original system, how- 
ever this is not needed for controller design for an interesting class of problems 
as this work demonstrates. 

Suppose that at time k, TT{x{k)) € XjE^^ is known. In the case when the 
estimates of the state at time k provide sufficient information to uniquely de- 
termine the membership of the state of the induced system at time A: -I- 1 on an 
equivalence class of E^r, the system is said to be quasideterministic. The notion 
of quasidetermism is illustrated in Figure 3. Although we do not compute an 
equivalence relation that guarantees the existence of a mapping / that preserves 
the reachability properties of the original system, we exploit the commutativity 
of the diagram (c) in Figure 3 in order to analyze the reachability properties 
with respect to the control specifications. The formal definition for the concept 
of quasideterminism is given in later in the section. 

Denote by B{X) the set of all binary relations on the set X. We can define 
the poset (E(X),<) where the partial order relation < on B{X) is defined as 
Bi < i ?2 if (xi,X 2 ) & Bi ^ (xi,X 2 ) & B 2 - Let E{X) be the set of all equivalence 
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-X/E„ X/E, 



(a) Primary Partition 




(b) Final Partition 




(c) Quasideterministic Partition 



Fig. 3. Quasideterminism and the partitions of the state space 



relations on X. We have that E{X) C B{X) and E{X) inherits the partial 
order of B{X). A lattice structure can be developed on the set of all equivalence 
relations on X (for more details see [10]). The lattice (E{X),<,A,V) is called 
the equivalence lattice. 

Proposition 1. The set Ep{X) of all equivalence relations on X induced by 
mappings tt : X ^ XjE^^ which are defined using finite collections of {n — 1)- 
dimensional hyperplanes and thus, they separate the state space X into polyhedral 
equivalence classes, is a sublattice of the equivalence lattice E{X), and will be 
called polyhedral equivalence lattice. Furthermore, Ep{X) is not complete. 

Definition 2. The hybrid system {X, U, D,Y, M; /, g, m) with primary and final 
partition defined by X/Et^ and XjE^^p is quasideterministic with respect to the 
primary partition if for every region of the final partition z G XjE^rp and for 
all states x € X such np^x) = z, there exists unique region of the primary 
partition y G XjE^ such that y = Tr{f{x,u,d)) for every control action u € U 
and exogenous input d G D. 

If the hybrid system (X, U, D,Y, M; f, g, m) with primary and final partition 
defined by XjE^ and XjE^p is quasideterministic with respect to the primary 
partition tt, then it is also quasideterministic if instead of E^p we use any finer 
final partition E^^ < E^p . Refinement of the state space partition will terminate 
if we can guarantee that there is a control policy to satisfy the specifications. 

4.3 Partition Refinement 

In the following, we present some basic results that will be used in the theoretical 
analysis of the algorithms for the partition refinement. A piecewise-linear (PL) 
subset [12] of a finite dimensional vector space V is the union of a finite number of 
sets defined by (finitely many) linear equations f{x) = a and linear inequalities 
f{x) > a. An alternative way to define PL sets which is important for our 
discussion is the following [12]. 

Definition 3. Let C be the first- order language defined by (i) a set of (countably 
many) variables {xi,X 2 , . . .}, (ii) the connective symbols ^ and (Hi) the 
quantifier V, the parentheses ( and ) and the comma, (iv) A set of constants 
{r} for each real number r, (v) A set of unary functions {r ■ ()} for each real 
number, the binary function +, (vi) the relational symbols > and =. 
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Lemma 1. Every sentence in L defines a PL set and conversely, every PL 
subset of K" can he defined in this fashion. 

The above lemma is proved in [12]. The conclusion of the lemma is that 
any set defined using quantifiers can be also defined using only propositional 
connectives. In order to refine the state space, we define the predecessor operator 
pre : P(X) — > P(X) as 

pre{P) = {x\3u G U,yd € D, f{x, u, d) G P}. (6) 

The set pre{P) represents all the states x for which there is a control action 
that will enforce the state to remain in P for any disturbance d. If the set P 
is piecewise-linear, then from Lemma 1 it follows that the set pre{P) is also 
piecewise-linear and can be defined using only propositional connectives. 

In the remaining of the paper, we will concentrate on the case the hybrid 
system is described by 

{X,U,D,Y,M;f,g,m) (7) 

with finite input set U, bounded disturbance set D, and transition function given 
by x{k + l) = Ax{k) + Bu{k) + Ed{k) . Similar results can be developed for other 
classes of piecewise-linear systems. 

Initially, assume that the state transition function is given by a; (/c -I- 1) = 
Ax{k) + Bu{k) where x G M" and the input u takes values in a finite set U C R™. 
For fixed control action u G U the dynamics of the system are described by the 
mapping /„ : R” — > R” with fu{x) = Ax + Bu. We want to compute the set of 
all the state x that can be driven in P by the control action u by defining the 
predecessor operator prcf,^{P) = {a;|/„(a;) = Ax + Bu G P}. 

Lemma 2. Consider the affine function h{x) = — w and the set H = 

ker(/i) = {x\g^x — w = 0}. Let H' = {x\fu{x) = Ax + Bu G H} be the set of all 
x G R" that can he driven in H by application of the affine mapping fu ■ Then 
H' = ker(h') where h'{x) = g^x — w' with g^ = g'^A and w' = w — g"’" Bu. 
In addition, ifY = int{K') is an open halfspace hounded by H' , then fu(Y) = 
int{K), that is fu{Y) is an open halfspace hounded by H. 

Next, we define the halfspace P{g, w) = {x\g^x < w}, g 0 and we compute 
the set of all states that can be driven to P by using the predecessor operator 
prc 3 : P(X) ^ P(X) defined as pre^{P) = {x|3u G U, fu{x) = Ax + Bu G P}. 

Lemma 3. Consider the set P{g,w) = {x\g'^x < w}, g yf 0}, then pre^{P) = 
{x\g"'"Ax < w — g^ Bu*} where u* is the maximizer of the function w{u) = 
w — g"’" Bu over the set of control actions Lf . 

Let / : AT — > F be a mapping and consider the sets D C X and E C Y. 
The image of D and the inverse image of E under the mapping / are defined 
by f{D) = {f{x)\x G D}, f~^{E) = {x\f{x) G E}. It is easily verified that the 
map f~^ : P(F) ^ P(X) commutes with unions, intersections, and complements. 
The operator pre/„ : P(A1) ^ P(X) {X = R”) clearly returns the inverse image 
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of P under the mapping /„ for fixed input and therefore commutes with unions, 
intersection, and complements. The notation pre/„ has been used instead of f~^ 
in order to be consistent with the notation when the control action is not fixed. 
In the case when the input set is finite, the set pre^{P) can be computed for 
any PL set as the union U« (-^)- 

Next, we consider the case when continuous disturbances are present and we 
assume that for a fixed discrete control action the description of the system is 
x{k + 1) = Ax{k) + Bd{k) where x € M” and d G D C a disturbance which 
takes values in a bounded polyhedron. We define a new predecessor operator 
pre'j : P(A") ^ P(X) by prej{P) = {x\\/d € D,f{x,d) = Ax + Bd S P}. This 
operator returns all the states which will be in the set P at the next time step 
for every possible disturbance. 

Lemma 4. Consider the set P = P{g,w) = {x\g"'"x < w}, then pre'j{P) = 
{x\g^Ax < w — g^ Bd*} where d* = argminj^^jj{— g^ Bd} . 

The predecessor operator in the case of bounded disturbances commutes 
with the intersection of halfspaces. Note that this result is a consequence of the 
equivalence iyx){4>{x) A 'ip(x)) ^ (Va;)^(a;) A {yx)'ip(x) in predicate logic. 

In the following, we consider the system x{k + 1) = Ax{k) + Bu{k) + Ed{k) 
where the disturbance d takes values in a bounded polyhedral set D and the 
control input u takes values in a finite set U and the polyhedral set P = {x\gjx < 
wi A ■ ■ ■ A gp < Wp}. Then by using the results of this section we have that 

pre{P) = {x|3m € U,\/d G D, f{x, u, d) — Ax + Bu + Ed £ P} 

= U (P) 

UiGU 

= {x\gf Ax < wi — g\ Bui — gi Edl A • • • A g^ Ax < Wp — g^ Bm — g^ Ed},} 

Ui£U 

where d* = argmaxd^D{—gJ Bd} . Next, consider the hyperplanes h'^{x) = 
gj Ax — (wi — gf Bui — gfBd}), i = l,...,p and the partition tt' G Ep(X) 
defined by those hyperplanes using Equation (3). 

Proposition 2. The hybrid system (7) with primary and final partition defined 
by XjE^^ and inf P,r') respectively is quasideterministic with respect to the 
primary partition. 

The implication of the above proposition is that for every state, every control 
action, and every disturbance the membership of the state at the next time step 
to an equivalence class of the primary partition can be uniquely determined from 
the current region of the final partition. Given a fixed time window repetitive 
applications of the predecessor operator can take into consideration more than 
one time steps. At this point it is possible to construct a discrete-event system 
based on the final partition irp and extend supervisory control techniques in 
order to exploit the information that is preserved in the discrete abstraction due 
to quasideterminism. However, we continue with our analysis of specific control 
problems for which we can formulate conditions for the existence of control 
policies that guarantee that the specifications are satisfied. 
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4.4 Safety 

In the following, we focus on the safety problem and we describe algorithms 
for the refinement of the state space partition that result in quasideterministic 
systems. Given a set of safe states described by the piecewise-linear set P C M” 
and an initial condition xq = x(0) G P, we say that the system is safe if x{k) G P 
for every k. The system is safe with respect to the set P if 

P C pre{P) = {x\3u G C/, Vd G D, f{x, u, d) G P}. (8) 

The validity of equation (8) can be tested using the representation of pre{P) 
without quantifiers. Since the set pre{P) is piecewise-linear but not polyhedral, 
the development of efficient algorithms that test if the equation (8) holds is 
necessary and is a topic of current research. A simple algorithm to perform this 
test consists of representing the complement of pre{P) as the union of polyhedra 
Q = [pre{P)Y = Ui=i p Qi then, testing if PnQi = 0 for every z = 1, ..., p 
using linear programming techniques. A simple way to express Q as the union of 
polyhedra is to consider all the inequalities that define Q pairwise and eliminate 
all the pairs that correspond to parallel hyperplanes. 

Proposition 3. Given the polyhedral set of safe states P and the hybrid system 
(7), if P n Qi = %,i = l,...p where Q = [pre{P)Y = lJi=i pQi’ then there 
exists control policy that guarantees that the system is safe. 

Example - Automotive Suspension System The automotive suspension system is 
safe if the chassis level is inside the interval [sp — It, sp + hi] while in straight 
driving mode. Our approach for the design of the controller is that given the 
desired-set point and therefore the primary partition, a final partition can be 
constructed and the conditions of Proposition 3 can be tested in an autonomous 
manner. If there exists a control policy that guarantees that the system is safe, 
then a controller that implements such a policy can be designed based on the 
discrete abstraction induced by The same approach can be used also 

off-line to characterize all the set-points for which there exists a control policy 
that guarantees safety. 

In order to construct the final partition, we translate the control specification 
from the output space to the input space to obtain the set Pi = {{xi,X 2 )\sp—lt < 
xi < sp + ht}. Clearly, the set P\ is unbounded in the state space From 
Lemma 2 it follows that the set pre{Pi) is bounded by hyperplanes that in gen- 
eral intersect with Pi and therefore, it is not possible that Pi C pre(Pi). The 
practical implication of this observation is that if the chassis level is very close 
to the boundary of the set Pi, then if the chassis vertical velocity is large and 
directed towards the unsafe region, there will be no finite control input that 
will guarantee safety. In order to proceed with the controller design we have 
to determine a bounded approximation of the set Pi by taking into considera- 
tion realistic bounds for the chassis vertical velocity. The final partition can be 
determined using the partition refinement algorithms described above. The pri- 
mary and final partition for typical values of the system parameters are shown 
in Figure 4 where it can be seen that P C pre{P). 
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Fig. 4. Final partition 



5 Hybrid System Regulator 



In this section, the regulator problem for hybrid systems is formulated. In gen- 
eral, a regulator requests certain types of outputs from the plant so that these 
are attained in the presence of disturbances. The desired outputs are charac- 
terized by a regulation condition and they can be described as the outputs of 
another SDS, called the exosystem. The plant and the exosystem are linked by a 
controller to form a regulator as shown in the Figure 5(i). A feedback controller 
can be designed to regulate the system. The main characteristic of the controller 
is that it contains a copy of the exosystem in accordance to the “internal model 
principle” . 

In the following, we consider the safety problem and we describe how a 
controller can be designed based on the discrete abstraction induced by the 
final partition. The state of the controller correspond to the regions of the 
final partition and the current state Xc = can be determined by fil- 

tering the plant measurements using the inequalities that define the equiva- 
lence classes of the final partition. The controller can be described by the SDS 
C = (Ac, Y X M, U; fc, gc) where Ac is the state set of the controller; Y x M is 
the input set of the controller consisting of pairs describing the output request 
and that actual plant output every time instant; U is the output set representing 
the control actions; fc ■ Ac x (A x M) Ac is the state transition function 
for the controller; and gc '■ Xc x {Y x M) ^ U is the output function given by 
u = gdxc, {rne{xe,v),TTF{x))). Since for some states there exist more than one 
control inputs that can be applied for safety, there are several ways to imple- 
ment the output function of the controller. For example, the output function can 
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defined by 



' Mo 
Ml 



if a; G Po = 

if X G Pi = prej^^ \ Pq 



un if X G Pn = \ Pn-i 



(9) 



Example - Automotive Suspension System The controller for the automotive 
suspension system is shown in Figure 5(ii). For the straight driving mode the 
controller is represented as a finite automaton with three different states corre- 
sponding to the regions of the final partition for the set P and output function 
defined by (9). For the curve driving mode, the controller consists of one state 
with constant output function m = 0. The controller communicates with the 
plant and the exosystem in a synchronous manner. 




Fig. 5. (i) Hybrid system regulator, (ii) Controller 



Remark A problem related to safety is to examine if there exists a control policy 
that will drive the state of the system to a prescribed region. For example, since 
at the end of a curve the chassis level may not be inside the interval [sp— It, sp + 
ht], it is required that as soon as the system is in straight mode the chassis level 
must be driven to the safety region by using either the compressor or the valve. 
This is a reachability specification that can be also studied in the framework 
presented in the paper. The final partition can be constructed by repetitive 
applications of the predecessor operator. For the termination of the partition 
refinement algorithm, the reachability specifications should be characterized by 
bounds on the time for the state to reach the desired region. 

6 Conclusions 

A novel hybrid control synthesis approach is demonstrated using an automo- 
tive suspension system. Controller design is based on quasideterministic discrete 
abstractions of the continuous dynamics. The regulator problem for hybrid sys- 
tems is formulated for safety specifications and algorithms for control design are 
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presented. Although a second-order system was used the approach, the method- 
ologies and the algorithms described are applicable to more complex systems. 
The approach has been validated with simulations using Matlab, Simulink, and 
Stateflow but simulation results are omitted due to length limitations. An im- 
portant point is that the above approach is potentially implementable on-line 
for real-time control. Note that due to space limitations, detailed descriptions of 
the technical results were omitted, but they can be obtained by contacting the 
authors. 
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Abstract. This report describes the calculation of the reach sets and 
tubes for linear control systems with time-varying coefficients and hard 
bounds on the controls through tight external and internal ellipsoidal 
approximations. These approximating tubes touch the reach tubes from 
outside and inside respectively at every point of their boundary so that 
the surface of the reach tube is totally covered by curves that belong to 
the approximating tubes. The proposed approximation scheme induces a 
very small computational burden compared with other methods of reach 
set calculation. 

In particular such approximations may be expressed through ordinary 
differential equations with coefficients given in explicit analytical form. 
This yields exact parametric representation of reach tubes through fam- 
ilies of external and internal ellipsoidal tubes. The proposed techniques, 
combined with calculation of external and internal approximations for 
intersections of ellipsoids, provide an approach to reachability problems 
for hybrid systems. 



Introduction 

Recent activities to promote advanced automation of real-time processes have 
motivated new interest in the problem of reachability for controlled systems. 
This is also related to the problem of verification of hybrid systems [4] . Effective 
and implementable solutions to these problems must incorporate procedures for 
calculating reach sets and reach tubes for continuous-time systems [13]. Another 
demand for effectively performing such calculations comes from interval analysis 
in scientific computation. [11]. 

Among methods for reachability analysis are those based on ellipsoidal tech- 
niques, (see, for example [2], [3], [6]). Publications in this area were mostly 
concentrated on deriving a single equation that would produce a sub-optimal 
(with respect to volume) ellipsoidal approximation to the exact reach set. 
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However, it turns that ellipsoidal methods allow exact representations of the 
reach sets and tubes for linear systems through parametrized families of both 
external and internal ellipsoids (see [6]). But to ensure effective calculation, an 
important open question is how to effectively single out such families of tightest 
ellipsoidal approximations to the reach tube that would touch its surface or the 
surface of its neighborhood at every point, (both from inside and outside !) and 
would thus totally cover this tube. A crucial point in organizing the calculation 
is to indicate such a parametrized variety of curves along which the procedure 
could be realized recurrently in time, without having to calculate the solution 
“afresh” for every new instant of time. A positive answer to the latter problem 
is given in this presentation for both external and internal approximations. It 
removes an unnecessary computational burden present in other methods and 
also opens new routes for deriving adequate numerical error estimates and new 
methods for systems other than those treated here [16], [12], [14]. The suggested 
approach is particularly relevant for hybrid systems since it allows further prop- 
agation to systems with resets. ^ An application of the proposed techniques to 
the verification of hybrid systems is given in paper [1]. 

In this paper we deal with reach tubes for control systems with linear dynam- 
ics and hard bounds on the control. We study the following question : given 
a reach tube (or its e— neighborhood) and a smooth curve that runs along its 
surface, do there exist ellipsoid-valued external (internal) tubes that would con- 
tain (be contained in) the reach tube and touch the reach tube precisely along 
the given curve? The answer to this question is positive. However the proper- 
ties of the respective ellipsoidal tubes do depend strongly on the given curve. 
The “good” situation is when the given curve may be realized as a trajectory of 
the original control system. ^ The required ellipsoidal tubes are then generated 
by ellipsoid-valued maps which satisfy the semigroup property and thus gen- 
erate some generalized dynamical systems. Moreover, the approximating tubes 
are tight in the sense that there exists no other ellipsoidal tube that could be 
squeezed in between the approximation and the reach tube (for both external 
and internal ellipsoids) . Lastly, the parameters of the ellipsoidal approximations 
are described by fairly simple ordinary differential equations. The paper also 
indicates the properties of the basic equations (18), (24) that allow them to be 
used correctly, without misunderstanding. Thus, it may be shown that when 
given is any smooth curve on the surface of the reach tube, which is not itself a 
system trajectory, there again exists ellipsoidal tubes that touch the reach sets 
along this curve. But now the respective ellipsoidal- valued maps may not satisfy 
the semigroup property and their evolution in time is not described by equations 
as simple as in the “good” case. The calculations then cannot be realized recur- 



^ These questions as well as the internal representations given here were not discussed 
in book [6]. 

^ This happens when the given curve (a system trajectory) develops along the points 
of support for hyperplanes generated by vectors that are realized as the motions 
of the linear system adjoint to the homogeneous part of the control system under 
investigation. 
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sively. They require procedures that have to memorize additional items and are 
therefore computationally heavier than in the “good” case. A simplification of 
the computational procedure in this general case to the level of the “good” case 
results in non- tight approximations(!). 



1 The Reachability Problem 

Consider the linear system 

X = A{t)x B{t)u, to<t<ti, (1) 

where x G M" is the state and u € R'" is the control. The matrices A{t),B{t) 
are continuous and the system is completely controllable (see [9]). The control 
u = u{t) is any measurable function restricted by hard bounds u{t) G V{t), for 
almost all t, where V{t) is a nondegenerate ellipsoid continuous in t, namely, 
V{t) = S{q{t),Q{t)), and 

S{q{t),Q{t)) = {u : {u - q{t),Q-^{t){u - q{t)) < 1}, (2) 

with q{t) G R™ (the center of the ellipsoid) and positive definite matrix function 
Q(t) G (the matrix of the ellipsoid) continuous in t. The support function 

of the ellipsoid is 

p{l\S{q{t),Q{t))) = ma.x{{l,x)\x G S{q{t),Q{t)} = {l,q{t)) + {l,Q{t)lf/'^ . 

The continuity of Q{t) means that its support function p{l\Q{t)) is continuous 
in t uniformly in I with (1,1) < 1. 

Definition 11 Given position the reach set (or “attainability do- 
main”) at time t > to from this position is the set 

X[t] = X{r,to,x^) = {a;[T]} 

of all states x[t] = x(t, to, x^) reachable at time r by system (1), with x{to) = x^, 
through all possible controls u that satisfy the constraint (2). The set-valued 
function t ^ X[t] = X{r,to,x^) is known as the reach tube. 

The reach set X{T,to,X°) (at time t, from set X^ = X(to)) is the union 

X{T,to, X^) = U{A(r, to,a^°)k° G A®}. 

The set-valued function t X[t] = X{T,to, Xq) is known as the reach tube 
from set A°. 

The following properties may be checked directly. 

Lemma 1. The set-valued map X{t,to, X^) satisfies the semigroup property 



X{t,to,X°) = X{t,T,X{T,to,X°)). 



(3) 
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In the sequel it is assumed that = £(x^, is an ellipsoid. It is worth noting 
that the set X[t] may also be treated as the cut X[t] = X{T,to,S{x^,X°)) of 
the solution tube X{-) = {X[t] : t > to} to the differential inclusion 

X & A{t)x + E{B{t)q{t),B{t)Q{t)B' {t)), t > to, G f (4) 

A standard calculation using convex analysis indicates the following (see, for 
example [6]). 

Lemma 2. The support function 

p{l\X{t, to, S{x°,X°)) = {I, x^{t)) + {I, X{t, to)X°X'{t, to)0'/"+ (5) 

+ f {l,X{t,s)B{s)Q{s)B'{s)X'{t,s)iy/^ds. 

Jto 

Here X(t, s) is the transition matrix for the homogeneous system (1), 

dX{t,s)/dt = A{t)X{t,s), X{s,s) = I, x* = A{t)x* + B{t)q{t), x*{to) = x^, 

where / is the identity matrix. For a time-invariant system A(t) = A = const, 
and X{t, s) = exp(A(t — s)). The last representation leads to the next result. 

Lemma 3. The reach set X\t] = X{t,to,S{x^,X^)) is a convex compact set in 
R” that evolves continuously in t. 

Points on the boundary of the reach set X[t] have an important characteri- 
zation. Consider a point x* on the boundary 9 A’[t] of the reach set X[t] = 
X{T,tt^,£{x^ , X^))f Then there exists a related support vector I* such that 

{l*,x*)=p{l*\X[T]). (6) 

The control u = u*{t) and the initial state x{to) = x*^ G £{x^,X^) which 
transfer system (1) from state x{to) = x*° to x{t) = x* is specified by the 
well-known “maximum principle” (see details in [9]). However, the calculation 
of the reach sets directly from these relations, especially in large dimensions, 
is cumbersome. Among the effective methods for these problems are those that 
rely on ellipsoidal techniques, as given in [6]. 

Remark 1.1 Due to the controllability assumption we will further assume, with- 
out loss of generality, that B{t) = I. To return to the case B{t) ^ I it suffices in 
the sequel to substitute everywhere Q{t) by B{t)Q{t)B'{t). However, in the last 
case, for computational purposes it may be useful to start the approximation 
process at time t = to + <^, > 0, to have W (to + 5, to) > 0. 

® The boundary dA’[r] of set X[t] may be here defined as the set dA’[r] = 
X[t] \ intX[T], Under the controllability assumption, set X[t\ has a non-void in- 
terior mtA’[r] A 0 for r > to. 
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2 Ellipsoidal Approximation of Reach Sets 

Although the initial set S{x^ , A°)) and the control set S{q{t),Q{t)) are ellipsoids, 
the reach set X[t] = X{t,to, S{x^,X^)) will not generally be an ellipsoid. As indi- 
cated in [6] , the reachability set X [t] may be approximated both externally and 
internally by ellipsoids £- and £+, with £- C X[t] C The approximations 
are said to be tight if for any ellipsoid £ the inclusion X[t] Q £ C £^ implies 
£ = £+, while inclusion C £ C X[t] implies £ = £-. Here we shall deal with 
both tight external and internal approximations. 

Problem 2.1. Given a vector function l*{t), {I*, I*) = 1, continuously differen- 
tiable in t, find external and internal ellipsoids £^[t] C A[t] C £^\t] such that 
for all t > to, the equalities 

p{r{t)\x[t]) = p{i*{t)\£4t]) = p{i*{t)\£-[t]) = {i*{t),x*{t)), (7) 

hold, so that the supporting hyperplane for X[t] generated by l*{t), namely, the 
plane {x — x*{t),l*{t)) = 0 that touches X[t] at point x*{t), is also a supporting 
hyperplane for [t] , 51 [t] and touch them at the same point. 

The solutions to this problem are given within the following statements. 

Theorem 21 With l{t) = l*{t) given, the solution to Problem 2.1(extemal) is 
an ellipsoid £+[t] = £{x*(t),X^[t]), where 

Xl\t] = p;(s)ds+Po(t)^ 

(^j\p*{s))-^X{t, s)Q{s)X'{t, s)ds + p*o-\t)X{t, to)X°X'{t, to)) , (8) 

and 

pt(s) = {l*{t),X{t, s)Q{s)X'{t, s)l*{t)y/^ (9) 

Poit) = {l*{to),X{t,to)X°X'{t,to)l*{to))^/^. 

This result follows from [6], [7]. Since the calculations have to be made for all 
t, the parametrizing functions pt{s),s G [to,t],po(t) must depend on t. Note 
therefore that the result requires the evaluation of the integrals in (8) for each 
time t and vector 1. If the computation burden for each evaluation of (8) is C„t, 
and we estimate the reach tube via (8) for T values of time t and L values of I, 
the total computational burden would be CnTL. 

In other words, relations (8), (9) need to be solved “afresh” for each t. It may 
be more convenient for computational purposes to have them given in the form 
of recurrence relations. As indicated further, in the next Section, this could be 
done by selecting function l*{t) of Problem 2.1 in an appropriate way. 

A similar result is available for internal approximations. 

Theorem 22 With I = l*{t) given, the solution to Problem 2.1 (internal) is an 
ellipsoid £{x-{t),X-{t)) , where 



X.{t) 



( 10 ) 
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= X{t, to) X{to, T)Q^/^{T)S'{T)dr^ 

(^Sot{to)Q"J^ + St{T)Q^/\T)X'{to,T)'^X'{t,to). ( 11 ) 

with So, Stir) satisfying relations 

5 t(r)gi/ 2 (r)X'(f,r)r(f) = \t{T)SotQT X' {t,to)l* {t), ( 12 ) 

and S'otSot = I] S[{T)St{T) = I for all t>to,rG [to,t], where 

Ai(r) = (r (t), X{t, T)Q{T)X'{t, T)l*{t))^/\r{t),X{t, to)QoX'{t, to)l*{t))^'\ 

( 13 ) 

The parametrizing functions are orthogonal matrix-valued functions St (t) , Sot ■ 
They too are dependent on t, so that the calculations have to be done “afresh” 
for each t as in the “external” case. Thus, the computaion in general is not 
recursive. To ease the computational burden we look for recurrence relations. 



3 Recurrence Relations 

There is a special selection of functions l*{t) that lead to recurrence relations. 

Assumption 31 The function l*{t) is of the form, I* ft) = X{to,t)l , with I G 
R” given. For the time-invariant case l*{t) = e~^ 

Then p’f{s),Po(t),Xf[t] of (9), (8) transform into 

pI{s) = {I, X{to, s)Q{s)X'{to, s)0'/" = P*{s); p*{t) = {I, X°l)^/^ = p*, (14) 

and 

Xl[t]=X{t,to)X+{t)X'{t,to), X+[t] = ^ p*(s)ds + p5^tf'(t), (15) 

where 

m = ( 16 ) 

= f {l, X{to, s)Q{s)X\to, s)l)-^/^X{to, s)Q{s)X'{to, s)ds + {I, X°l)~^/^X°. 
Jto 

In this particular case pf{s) does not depend on t (pf(s) = Pt”(s) for f yf t”) 
and the lower index t may be dropped. 

^ Under this Assumption the vector T(t) is the solution to equation 

I* = -A'{t)l*, l*(to) = l, 



which is the adjoint to the homogeneous part of equation (1). 
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Direct differentiation of X^[t] yields 

X+[t] = 7T*{t)X+[t] + 7T*-\t)X{to,t)Q{t)X'{to,t), X+[to] = x°, (17) 



where 



Calculating 



7T*(t) p*{s)ds + pg 



{l,X+[t]l)= ( f p*{s)ds+p^]{l,mi) = if P*{s)ds+p*o)\ 
\Jto J Jto 

one may observe that 

7r*(t) = il,X{to,t)Q{t)X'{to,t)l)^^^{l,X+[t]l)-^/^. 

In order to pass to the matrix function X’^ [t] we note that 

Xl[t] = A{t)X{t,to)X+[t]X'{t,to) + X{t,to)X+[t]X'{t,to)A{t) 
+X{t,to)X+[t]X'{t,to)- 

After a substitution from (16) this gives 

a; = A{t)Xl + A;A'(t) + 7r*(t)A; + TT*-\t)Q{t), X*{to) = x°. 
We summarize these results as follows. 



(18) 



(19) 



Theorem 31 Under Assumption 3.1 the solution to Problem 2.1 (external) is 
given by the ellipsoid = £{x+{t), X)_[t]), where x+(t) = x*(t) and is 

a solution to equations (18), (16). 

Since the set depends on vector I, we denote X)_[t] = X^[t]i. 

Theorem 32 For any t > to the reach set X\t] may be described as 

X[t] = n{£(x+(t),A;[t]0}| I : {1,1) = 1}. (20) 

This is a direct consequence of Theorems 3.1. 

Thus, if l*{t) satisfies Assumption 3.1, the complexity of computing a tight, 
external ellipsoidal approximation to the reach set for all t, is the same as com- 
puting the solution to the differential equation (18). If L values of I and T values 
of t are evaluated, the computational burden is C„TL. 

For the general(non-recursive) case, the relation corresponding to (18) is far more 
complicated and is actually a functional-differential equation which requires re- 
calculations for each t. If however (18) is still used for the general case, the 
inclusion X[t] C £+[t] remains true but the tightness property is lost. 
Throughout the previous discussion we have observed that under Assumption 
3.1 the tight external ellipsoidal approximation £{x*,X)_{t)) is governed by the 
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simple ordinary differential equations (18). Moreover, in this case the points 
X* (t) of support for the hyperplanes generated by vector I (t) run along a system 
trajectory of (1) which is generated by a control that satisfies the maximum 
principle. 

Similar facts are also true for internal approximations. We now again select 
function l*{t) to satisfy Assumption 3.1. Then substituting l*{t) in (12), (13), we 
observe that the relations for calculating Stir), Xtir) transform into 

5t(r)Qi/2(r)X'(to, t) 1 = Xt{T)SotQ"J^l; S'^So = /; 5'(r)5(r) = I (21) 

and 

At(r) = {I, X{to, T)Q{T)X'{to, T)iy^V{l, Qoiy^^. (22) 

Here the known functions used for calculating Stir), Xt{T) do not depend on t. 
Therefore, the unknown functions St{r), Xt{r) do not depend on t either, no mat- 
ter what is the interval [to, t]. The lower indices t in Sot, St, At may be dropped. 
Differentiating (10) in view of the last remark, we come to 

X— = A(^t)X— X—A'(^t) -l- Q'sfQ*, (23) 

where 

Q,{t) = SoQy"x\t,to)+ [ 5(r)Qi/"(r)X'(t,r)dr, 

Jto 

Q*{t) = S{t)Q^^^{t),Q^{to) = SoQo- 

Using the notation 

H{t) = Q:\t)S{t)Q^/^t) = Q:^{t)Q4t), (24) 

we further come to equation 

= A{t)X_ + X_A\t) + H'{t)X_{t) + X_{t)H{t), AT(to) = Qo- (25) 

and also observe that the center X-{t) = x+{t) = x*{t). This leads to the 
following theorem. 

Theorem 33 Under Assumption 3.1 the solution to Problem 3.1 (internal) is 
given by ellipsoid S{x-(t), X-(t)) where X-{t) is given by equations (24), (23), 
and the functions S(t),X(t) involved in the calculation of H{t) satisfy together 
with So the relations (20), (21), where the lower indices t in Sot,St,Xt are to be 
dropped. 

Function H{t) = Qf^{t)S{t)Q^/'^{t) in (23) may be also expressed through equa- 
tion 

Q* = Q,A'{t) + S{t)Q^/^{t), Q4to) = SoQI^^. (26) 



This gives the result 
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Lemma 4. The ellipsoid E{x-{t),X-{t)) of Theorem 3.3 given by equations 
(23)-(25) depends on the selection of the orthogonal matrix function S(f) and 
for any such S(t) the inclusion 

S{x-{t),X_{t)) C X[t], t>to, (27) 

is true with equalities (7)(internal) attained under conditions (20), (21). The 
following relation is true 

X[t] = cl{yj{£{x.(t),X_[f\i)}\ I : {1,1) = 1}}. 

where clY stands for the closure of set Y. 

The boundary of X\t\ is thus described as a function of a finite- dimensional 
parameter I € R”. 

Let us now suppose that function l{t) of Problem 2.1 (internal) is any continuous 
curve on the surface of X[t\. Then one has to use formula (10), keeping in mind 
that Sot,St{r) do depend on t. After a differentiation of (10) in t, one may 
observe that (25) transforms into 

X_ = A{t)X_+X_A\t) + Hft)X_{t)+X_{t)H{t)+<P{t,-), X^to) = Qo. (28) 

where ^(t, •) is a functional of Stir), Sot- The calculations are then far more 
cumbersome than under Assumption 3.1. If in this general case we still use 
the simpler equation (24), then the inclusion (26) will still be true, but the 
property of tightness will be lost. Note that under Assumption 3.1 the term 
<P{t, •) disappears. 



4 The Reach Tube 

The results of the previous Sections may be thus summarized as follows. Suppose 
Assumption 3.1 is fulfilled, then the points x*{t) of support for vector l*{t) = 
X'{t,to)l, I G R", namely, those for which the equalities 

{l*{t),x*{t)) = p{r{t)\X[t]) = p{l*{t)\£{x^t),Xim (29) 

are true for all t > to, may be reached from initial state 

vOj 

= = + (30) 

and from a trajectory x* (t) that satisfies the following “maximum relation” : 

{1* (t) , X* {t)) = max{(r(t), a;)|a; e £{x*, A^[t])}, (31) 

which is attained at 



X* (t) = a;* (t) + a; [t] I* {t) {I* {t) , XI [t]l* (t)) , 



(32) 
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where X^[t] = X[t] is the solution to equations (18), (17). 

For B{t) = I and Q{t) nondegenerate the same trajectory (31) may be attained 
through internal ellipsoids with 

x*it)=x*{t)+X_[t]r{t){l*{t),X_[t]l*{t))-^/^, (33) 

where X-[t] is a solution to (24), (23). The same property holds if Q{t) is 
nondegenerate and the system (1) is controllable. 

Denoting x*{t) = x[t,l], we thus come to a two-parameter surface x\t,l] that 
defines the boundary dX of the reachability tube X = U{Ar[t], t > to}. With 
t = t' fixed and I G S varying, (5 is a unit sphere), the vector x[t',l] runs along 
the boundary dX[t']. On the other hand, with I = I' fixed and with t varying, the 
vector x[t, I'] moves along one of the trajectories x*{t) that touch the reachability 
set X[t] according to (7). Then 

U{x[t,l]\l G 5} = dX[t], U{x[t,l]\l G 5, t>to} = dX 

Remark 4.1. The possibility of using both external and internal representations 
is important for treating hybrid dynamics for systems that allow resets. Thus, 
if for example set T® = one may introduce approximations of type C 
X^ C to start the calculations of the reach set X[t]. On the other hand, if 
for some t' > to we have X[t'] n Em, where Em stands for a given guard, we may 
introduce approximations of type 

£*-[t'] C u{E{x.{t'),x_{t'))\{i,i) < ijn^M 

c n{E{x+{t'),x+{t'))\{l,l) < i}n Em Q£+[t'] 

for the resets and proceed for t >t' with the procedures of Sections 2-4. 

5 An Example 

Taking system 

±i = X2, X2 = U, 

xi(0) = Xi,X2{0) = X2, |m| < fi, M > Oj '■ < e^}. 

and omitting the calculations, we indicate the external and internal ellipsoidal 
approximations of the respective reach set X\t] = X{t,0,X'^). 

Here the “good” curves of Assumption 3.1 have the form of straight lines: l*{t) = 
exp{—A't)l or 1} = li, t^i-They are shown in fig.l for e > 0. The external 

and internal approximations that touch the reach set X[t] along these lines are 
shown in fig. 2 and fig. 3 for e = 0 and in fig. 4 for e > 0. 

6 Conclusion 

This paper specifies and studies the behavior of the tight external and internal 
ellipsoidal approximations of reach sets and reach tubes for linear time-variant 
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control systems. It shows that equations (18), (24) with appropriately chosen 
parametrizing functions TT(t),S(t) generate two family of tight (external and 
internal) ellipsoidal aproximations to the reach tube X[i\ which touch it along 
a certain family of “good” curves that cover the whole tube. It gives analytical 
representations that allow to achieve a substantial reduction of the computation 
burden for calculating these sets as compared to direct methods and thus gives 
effective techniques for calculating the reach tubes in a compact recursive form. 
The analytical relations developed in this paper open routes to the investigation 
of precise error estimates in ellipsoidal approximations for problems of evolution, 
estimation and control as well as to the development of new computational tools 
for classes of systems more complicated than those treated in this paper. In 
particular, they indicate convenient tools for the treatment of hybrid dynamics 
(see [1]). 
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Abstract. We introduce the notion of a parametrized family {Hp)p^p 
of hybrid systems, and consider questions of reachability in the systems 
Hp as the parameter p ranges over P. Under the assumption of a uniform 
(as p ranges over P) finite bound on the number of discrete transitions 
associated to the individual systems Hp, the notion of reachability is 
first-order (in the sense of mathematical logic) and uniform in the param- 
eter p. Techniques from logic can then be used to analyze computational 
questions associated to the family of systems. 



This paper is concerned with uniform verification of reachability properties 
for parametrized families of hybrid systems. 

The central reachability question for a hybrid system (no matter how this 
is defined) is to determine, given two states of the system, whether there is a 
trajectory which takes the system from one state to the other. Ideally, one has 
an algorithm which takes as input pairs of states {x, y) and computes whether 
there exists such a a trajectory; see, for example, [1,5]. Tools from mathematical 
logic can be useful in these investigations. 

More generally, one can consider reachability questions for families of hybrid 
systems that are linked up in some reasonable fashion, and hope that one can find 
algorithms that work uniformly as one varies the systems under consideration. 
Most of the work in this paper goes into making precise statements of these loose 
notions. 

Here is an outline of this paper. In Section 1, we extend the definition of 
hybrid system given in [4,5] to that of a parametrized family of hybrid systems. 
Section 2 contains some relevant material from model-theoretic definability the- 
ory. We present the main results in Section 3, followed by some examples and 
applications in Section 4. 

1 Families of Hybrid Systems 

We begin with an informal discussion. The intuitive notion of a parametrized 
family of hybrid systems is fairly clear (once we have a clear notion of hybrid 

* Research supported by NSF grant DMS-9896225 



N. Lynch and B. Krogh (Eds.): HSCC 2000, LNCS 1790, pp. 215—228, 2000. 
@ Springer- Verlag Berlin Heidelberg 2000 




216 



G. Lafferriere and Ch. Miller 



system, that is). If we think of a hybrid system as some sort of black box, 
then a parametrized family (Hp)p^p of hybrid systems is a black box H with a 
console P = (Pi , . . . , P/ ) of dials Pi , . . . , P; such that each setting p= {pi,. . . ,pi) 
of the dials yields a hybrid system Hp. The dials control the various relevant 
sets, perhaps the vector fields involved, perhaps even the entire state spaces. 
Naturally, we would like the resulting systems to vary in some sensible way with 
respect to the settings of the dials. 

Similarly, the intuitive notion of uniform decidability (or computability, or 
whatever) of a family of hybrid systems is easy to describe (again, once we have 
a clear notion of decidability or computability of a hybrid system). For H as 
above, we should have algorithms, working uniformly over all settings p, for 
answering various questions about the systems Hp. For example, we should have 
some computable function <I>h such that: (a) given a setting p and states x, y of 
Hp, we have <pH{p,x,y) = 0 if and only if y is reachable from x in the system 
Hp] and (b) for a given pair (x, y) of possible states x, y, we can compute the set 
of all p such that x,y are states of Hp and (I>H{p,x,y) = 0. Further variations 
easily come to mind, but we should not digress too far at this point. 

We now begin to make these intuitive notions precise. First we carefully 
parametrize all data involved in the definition of a hybrid system, as given in [4] 
or [5]. Now, any set of hybrid systems can be made into a parametrized family: 
just index it set-theoretically by some suitable ordinal. This is a rather useless 
approach, of course. We incorporate in our definition a certain amount of desir- 
able uniformity. This is unavoidably rather tedious notationally, and we advise 
the reader to keep the informal discussion above in mind throughout. We stress 
that several other ways of doing this easily come to mind; some less complicated, 
some more complicated. We have chosen an approach somewhere in the middle. 

Of crucial importance in this paper are the notions of parametrized families 
of sets and maps. Let X,Y be sets, A C X x Y and x G X; then denotes 

the fiber of A over x, that is, the set { y G T : (x,y) G A}. (One can define 

similarly the fiber of A over y G Y, but we will not introduce notation for this.) 
The (first) projection tt{A) of A is the set of all x G X such that A^ yf 0. Given 
a map f : A ^ Z (with Z some set) and x G X, let f{x, • ) : A^ ^ Z denote 

the map y f{x,y) : A^ Z. Let B G_ X and consider the indexed families 

and (f{x,-) : A^ Z)x^b- The former is called a parametrized (by 
B) family of subsets of Y, while the latter is called a parametrized family of 
maps. (After identifying a map with its graph, a parametrized family of maps is 
just a special kind of parametrized family of sets.) Of particular interest is the 
case B = tt{A). 

Let M be a set and m,n gN. We identify M'" x M" with whenever 

convenient. (Regard as the one-point space {0}, and functions / : ^ M 

as the corresponding constant /(0).) Hence, given A C and x G M™, 

Ax Q Tf” denotes the fiber of A over x, and (unless stated otherwise) 7 t(A) 
denotes the projection of A on the first m coordinates. (We rely on context 
to indicate when subscripts indicate taking fibers, and when they are used as 
indices.) Moreover, for f = 1, . . . , m, we let denote the projection on the z-th 
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coordinate. (There will be times when, in order to avoid ambiguity, we shall have 
to abandon some of this notation, and just state things in words.) 



Examples, (a) Consider the set 

{ (a, b, c, d, e, /, x, y) G M® : ax^ + bxy + cy^ + dx + ey + f = 0} . 

As the tuple {a,b,c,d,e, f) varies over M®, we obtain the family of all conic 
sections in the plane, (b) Given a set ACM, let GL„(if) denote the set of all 
invertible n x n matrices with entries from E. Then 

( a; Ax : M" ^ M" )^eGL„(£;) 

is a parametrized family of maps. Note that a map A G GL„(E) can be identified 
with (or coded up as) as a point in R” , since the action of the map is determined 
by its coefficients. 

Definition 1. Let M be a set. A parametrized family of hybrid systems on M 
is a 5-tuple H = (M, S', /, T, F) where M , S, I, and E are sets, and F is a map, 
with the properties indicated below. 

— The set M is a Hausdorff, second countable, (sufficiently) differentiable man- 
ifold. 

— The graph space S is a nonempty subset of 

{(x,y, : X G G M™, l<i,j<m, ?/i < . . . < y™} C R"+™+^. 

The parameter space P is the projection of S on the first n-\-m coordinates. 
The projection of P on the last m coordinates is denoted by Q. 

— IQ R"+™ X M™, and its projection on the first n-\- m coordinates is equal 
to P. 

— PC R"+™+2 X M'^, and its projection on the first n + to + 2 coordinates is 
S. For a fixed z = {p,Pn+i,Pn-ej) G S, where 1 < i, j < m, we require that 

T^l{Pz) Q 7Ti(/p) and ■K2 {Pz) Q T^j{Ip)- 

— The map F : P x M ^ (TM)'^ is such that for each p G P, each component 
Fi{p,-) \ M ^ TM of F{p, ■) is a complete (i.e. trajectories are defined for 
all time) vector field on M. Here, TM denotes the tangent bundle of M . 

The (parametrized) flow of the map F : P x M ^ (TM)"^ associated to H is 
the function (j : P x R"* x ^ M™ defined by (j){p, t,x) = y if and only if for 
each i = 1, . . . , to, the integral curve of Tj(p, • ) with initial condition Xj passes 
through yi at time U. 

Each element of Q is an ordered list of vertices or locations for a single 
hybrid system. The continuous components of hybrid trajectories lie in sub- 
sets of M . For a fixed parameter p G P and the corresponding list of vertices 
q = (p„+i, . . . ,p„+m) = (gi, • ■ • , <7m) the projection nfllp) is nonempty and is 
referred to as the invariant set at location qi. The projection of S on the last 
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two coordinates is the set E of edges. For each z = {p^pn+i^Pn+j) G S, Ez de- 
fines a relation on M which induces discrete transitions (see below). The sets 
R{e) = {pn+i} X 7Ti(r'z) and G(e) = {pn+j} x 7T2 (Tj;) are respectively the reset 
and guard associated to the edge e = {pn+i^Pn+j)- 

Note that our definitions allow a parametrized variation of the discrete loca- 
tions (but not their number). 

Additional parametrized features could be added to the definition in an ob- 
vious way. For example, one may wish to specify some distinguished initial and 
final sets for trajectories (Go,G/ C ]^"+™+i x M, with suitable projections). 

While the present definition is concise, in special cases it may help intu- 
ition to have parameters separated into groups depending on which entities they 
parametrize: initial conditions, vector fields, invariant sets, and so on. 




Fig. 1. Parametrized family of hybrid systems 



Example. Figure 1 provides a schematic representation of a family of hybrid 
systems. This is encoded as follows, where a through /, rt, Si, pi, qi, and Ui all 
vary over appropriate sets of real numbers: 

- M = R2 

- P = {(a, 6, c, d, e, /, ri, T 2 , si, S 2 , mi, U 2 , vi,V 2 ,wi,W 2 , 1, 2)} 

- S = { {p,i,j) -.pe P, i,j = 1,2, z yf j } 

- Go = { (p, 1) : p G P} X [ri,si] x [zzi,ui] 

“ C*/ = { (P> 2) : P G P } X [r2, S 2 ] X [u2, U 2 ] 

- I = {{P, (Xl,yi), (X2,y2)) ■■ Xi < Wi,y2 > W2 } 

- P = { (p, 1, 2, {xi,yi), (x 2 , z/ 2 )) : = 0, a ;2 < -1, z /2 = 3 } U 

{ (p, 2, 1, (a;i, yi), {x 2 ,y 2 )) : a;i = -1, y = 1, z /2 = 0 } 

- F{p,x) = {A\x, A 2 X), where Ai are upper triangular matrices with entries 
a,b,c, and d,e, f respectively. 

Some restrictions on the parameters are needed to satisfy the inclusion require- 
ments (for example, wi > 0, ZC 2 < 0). In this example, parameters affect the 
vector fields and the initial and final sets, but do not influence the locations or 
the relation P. 
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For each fixed parameter p G P, H determines a hybrid system Hp similar 
to those introduced in [4,5]. Let Xd = {Pn+i, ■ ■ ■ ,Pn+m} be the set of discrete 
locations. Put Hp = (AT, Xq, AT/, Funct, Edge, Inv, Rel), where: 

— X = Xd X M is the state space. 

— Xo = {(<7, a:) € AT : 3 1 < i < m,with q = Pn+i, and {p,q,x) G Co\ is the 

set of initial states. The final states X f are defined similarly. 

— Funct : X ^ TM is defined by Funct(p„+i, x) = Fi{p, x) for each 1 < z < m. 

For each q G Xd, Funct((7, • ) defines a vector field on M. 

— Edge = Sp is the set of edges along which discrete transitions occur. 

— Inv : Xd — > 2^ assigns to each location the set Inv(p„_|_j) := iTi{Ip). 

— Rel = {{p„+^,x,p„+j,y) : z = (p,pn+i,Pn+j) G S,{x,y) G P^} defines dis- 
crete transitions. 

(For convenience, we omitted notation for the dependence of the new objects 
on p.) 

For each edge e = (q,r) define Rel(e) = {{q,x,r,y) G Rel}. The systems 
in [4,5] are a special case corresponding to Rel(e) = G(e) x R{e) (guard times 
reset as defined earlier). 

Consider the (single) hybrid system K = (AT, Edge, Inv, Rel, Funct). An ele- 
ment {{q,x),e,{r,y)) gXxExX is a, discrete transition (along e), denoted 
by (q,x) {r,y), if e = (q,r) and (q,x,r,y) G Rel(e). Fix some t ^ E. An 

element {{q,x),T,{r,y)) G X x {r| x A is a continuous transition, denoted 
by {q,x) (r,y), if q = r and there exist i5 > 0 and a differentiable curve 

7 : [0,(5] — > Inv((7) satisfying 7' = Funct((7, 7), 7(0) = x, and j(S) = y (that 
is, there is an arc of a trajectory of the vector field F{q,-) connecting x to y 
within Inv). Note that if (q,x) ^ {q,y) and (q,y) (q,z), then (q,x) ^ (q,z). 

Given a,b G X , we say that b is reachable from a (in K) if there exist k G N, 
(Ti, . . . , CTfe G E U {t| and states oq, . . . ,au G X such that a = ag, b = ak and 
Ui-i ^ Gi for z = I, . . . ,k. 

When we wish to emphasize the parameter set P we use the notation {Hp)p^ p 
to denote the parametrized family H . 

Given a family H = {Hp)p^p of hybrid systems, we define the reachability 
set of H, denoted by Reach(iL), to be the set of all (p,x,y) such that p G P 
and y is reachable from x in Hp. The reachability problem for H is just the set 
membership question for Reach(iL). 

Important Note. From now on, we will restrict our study to systems in 
which, for each e, Rel(e) is of a special form, namely, a finite union of subsets of 
Rel(e), each a cartesian product AxB C ({(7} x M) x ({rj x M), where e = {q, r). 
That is, we assume that for each edge e there is an integer zz(e) and sets Ai{e), 
Bi{e) as above such that Rel(e) = U”i}^(Ai(e) x Bi{e)). The results of [4] on 
bisimulations extend to this case with minor modifications. For such systems, 
since concatenations of continuous transitions collapse, there is an integer Nk 
such that, if b is reachable from a in K, then we may take k < Nk- Moreover, for 
a parametrized family {Hp)p^p we assume that the set |zzp(e) : e G E,p G P} is 
bounded above (i.e. the numbers np{e) are bounded uniformly in p). 




220 



G. LafTerriere and Ch. Miller 



2 Definability Theory^ 

We require some notions from model theory (a branch of mathematical logic) 
which, in its most general form, is the study of classes of models of theories 
in given languages, and the relationships between syntax and semantics. At its 
root, there are quite a few important — but rather tedious — technical definitions, 
creating pitfalls for the unwary outsider. To make matters worse, the subject 
has undergone something of a revolution in the last decade or so, resulting in 
changes of terminology, as well as entire points of view. Many of these recent 
changes have not found their way into standard texts. But there is a fairly small 
fragment of model theory that often suffices for applications to other subjects, 
especially for explaining and applying model-theoretic results: what can be called 
(first-order) definability theory. We present in this section a brief introduction 
to the subject. There are two equivalent approaches — informally, the top-down 
and bottom-up — each more useful than the other at times. 

We provide here neither history nor a comprehensive treatment of basic re- 
sults (and, for ease of exposition, we still gloss over some minor technicalities). 
Rather, our goal is to equip the reader with the basic technology necessary in 
order to understand how some current developments in model theory can be 
applied to hybrid systems. We also recast and clarify some material from [4,5]. 
The reader interested in historical context, original sources, detailed statements 
of results, proofs, and so on, may begin by consulting [15,16,17] for information. 



2.1 The Top-Down Approach 

In this scenario, we are interested in some particular class of sets that are (or that 
we hope are) closed under first-order definability; we make this notion precise. 

Let M be a nonempty set. A structure on M is a sequence Tl = (®l„)„gN 
such that for each n G N: 

— (SI) M” G dJln and is a boolean algebra of subsets of M” (that is, 
is closed under taking complements and finite unions). 

- (S2) { (a;i, ...,Xn) G M” : Xi = Xj} e 9H„, I < i < j < n. 

- (S3) If A G Mn, then Ax M, M x Ag Mn+i- 

— (S4) If A G dyin+i, then 7t(A) g DJln- 

We say that a set A C M” is definable in 9A, or that DR defines A, if A G 
DRn- If no ambient space M" is mentioned, then “definable set” (in DR) means 
“definable subset of M”, for some n G N”. A map / : A ^ M”, A C M™, is 
definable if its graph { (x,f{x)) : a: G A} C is definable. Whenever a 

particular structure DR is under consideration, we just say “definable” . 

The use of the word “definable” comes from a connection to first-order logic. 
Note the correspondence between the set-theoretic operations of complementa- 
tion, union, intersection and projection, and the logical operations of negation, 

^ This section is partially based on a lecture given by the second author at HSCC’99 
(Berg en Dal). 
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disjunction, conjunction and existential quantification. Closure under these log- 
ical operations is a very strong condition; see Appendices A and B of [17] for 
some examples of how this can be exploited. 

It is crucial to understand that definability is always taken with respect to 
some particular structure. Whenever we have more than one structure under 
consideration, we must take care to avoid ambiguities. 

Let TO,n € N, A C and B C M™. Then the parametrized family 

x^B is a definable family (of subsets of M”) if A and B are definable. 
If, moreover, f : A —> is a map, then the parametrized family ( f(x, ■ ) : 
Ax Mp)x^b is called a definable family of maps if / and B are definable. 
(The definability of A follows from that of /, since A is the projection on the 
first m + n coordinates of the graph of /.) Note that we can code up any finite 
collection of definable families as a single definable family (in the same structure) . 

There is a natural partial order on the class of all structures on M. Given 
structures dJl = (9Jl„) and = (®I(j) on M we put dJl C TV if C for 
all n G N. If C TR', then we say that (a) is a reduct of 911'; (b) TR' is an 
expansion of 991; or (c) 991' expands 991. 

Clearly, M has a largest structure on it: For each n G N, just let 99t„ be 
the collection of all subsets of M”. This is not a very interesting structure, 
but its existence is occasionally useful for theoretical purposes. There is also a 
smallest structure on M (also not very interesting). Usually, we are interested 
in structures that come equipped with some extra basic information. 

Let Sp C be sets (/? in some index set J) and fa ■ ^ M he 

functions (a in some index set I). A structure on ( M, (Sp), (fa ) ) is a structure 
991 on M such that each fa and each Sp is definable in 991. Equivalently, we say 
that 991 is an expansion of ( M, (Sp), (fa ) ). 

Given S C M, we say that A is S -definable (in 991), or definable with param- 
eters from S, if A is definable in (991, (c)ces), that is, in the expansion of 991 by 
constants for each c G S'. In the case S = M, we say that A is parametrically 
definable. Note that “definable” and “0-definable” mean the same thing. The dis- 
tinction between “definable” and “parametrically definable” is often extremely 
important in model-theoretic statements and arguments. 

In some branches of model theory, it has become more customary to use 
“definable” to mean “parametrically definable” (it’s more convenient for analytic 
and geometric purposes). But when computation is at issue, this is irksome: We 
don’t want to be involved with computing, say, arbitrary real numbers, and it 
doesn’t seem to make sense to talk about decision procedures (or algorithms) 
that range over uncountable collections of sets. When consulting the literature, 
one must determine in which sense “definable” is being used; when using the 
notion, one must take care to use it consistently. ^ 

Examples 

— Semilinear sets. Let AT be a subfield of K and U be a iL-linear subspace of 

M. For each n G N, let 99l„ be the collection of all finite unions of sets of the 
^ This has been a problem in some earlier papers on hybrid systems. 
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form 



{x G y” : fi{x) = ••• = fk{x) = 0, gi{x) < 0,...,gi{x) <0} 

where each fi and each gj are affine iG-linear maps V. (If K = Q, 

then we can take the coefficients of the maps to be integers.) It’s routine 
to check that each satisfies (SI)-(S3). Verifying (S4) takes a bit more 
work, but it’s not difficult; see e.g. pages 25-27 of [16]. 

— Semialgehraic sets. Let i? be a real-closed ordered field (M, for example). For 
each n G N, let be the collection of all finite unions of sets 



{a; G i?” : f{x) = 0, gi{x) < 0,...,gi{x) <0} 

where / and each gj are n-variable polynomial functions with coefficients 
from R. (If R is the field of real algebraic numbers, then we can take the 
coefficients to be integers.) It’s again routine to check that each collection 
satisfies (S1)-(S3). That (S4) holds is due to A. Tarski [13]; for an interesting 
alternate proof, due to S. Lojasiewicz, see Ch. 2 of [16]. 

— Subexponential sets. For each n G N, let be the collection of all projec- 
tions on the first n variables of sets { {x,y) G : F{x,y) =0}, where 

fc G N and F : ^ M is a function from the ring 



Z[xi, ...,Xn,yi,-- ■ 



*,e“ 



In this case, verifying (S2)-(S4), along with showing that these collections 
are closed under finite intersections, is the routine part. The (rather hard) 
work of showing that they are closed under complementation (hence also 
under finite unions) is due to A. Wilkie [18]. 

— Finitely (or globally) subanalytic sets. For each n G N, let be the collec- 
tion of all subsets of K" whose image under the map 



{xi, 



Xi 












is subanalytic. Here, again, closure under complementation is the hard part. 
The result is essentially due to A. Gabrielov [2]; see also L. van den Dries [14]. 
For applications of subanalytic geometry in control theory, see e.g. [12]. 



2.2 The Bottom-up Approach 

In this case, we are given the set M together with some functions on, and subsets 
of, various cartesian products, and we close off under definability. 

For each n G N, let Vn be a (possibly empty) collection of subsets of M” 
and Fn be a (possibly empty) collection of functions M” ^ M. Elements of 
Vn and Tn are sometimes called, respectively, primitive relations and functions, 
or just primitives. We now regard M as being equipped with these relations 
and functions, that is, we consider the structure ( M, (Vn), {Fn) ) as an algebraic 
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object. (We shall see that our use of the word “structure” for two formally 
different objects causes no trouble when we are concerned with definability.) 

For each n G N, let Tjj be the smallest set of functions on M” such that: 

(a) TjjContains the coordinate projections tt^ : M” — > M for i = l,...,n; and 

(b) for all m eN, F & and fi, ■ ■ ■ , fm & %i, we have F o (/i, . . . , /^) G 7^. 
We construct collections dJln,k of subsets of M” by induction on fc G N. For 

k = 0 and n G N, let lXftn,o be the boolean algebra generated by the collection of 
all sets of the following forms: 

{x e M'^ : f{x) = g{x)}, f,gG%, 

{ a; G M" : (/i(a;), . . . , fm{x)) G P}, f P G Vm, m G N 

Assume that the stage k collections have been constructed. For n G N, let dJln,k+i 
be the boolean algebra of subsets of M” generated by U { tt{A) : A G 

}• 

For n G in, put := It’s easy to see that is the 

smallest structure, in the top-down sense, on ( M, (Pn), (Fn ) ); so we just denote 
it by {M,{Vn),{Fn)) and call it the structure on M generated by (Vn), (Fn)- 
(Often, for convenience, we just list the primitives.) And, of course, we say that 
A C M” is definable in ( M, (P„), (F„) ) if A G Tin- 

Let Tft = (Tin) be a structure on M in the top-down sense. Clearly, a set A is 
definable in Tl in the top-down sense if and only if A is definable in ( M, (®l„) ) 
in the bottom-up sense. 



Examples 

— The sets definable in (Q, <, -b, — , 0, 1) are the (rational) semilinear sets. Here, 
the symbol — denotes the function x i-^- —x : Q ^ Q. The function — and 
the constant 0 are definable in (Q, <,-b,l), but often we include them as 
primitives for convenience. On the other hand, 1 is not definable in (Q, <, -b). 

— If i? is the set of all real algebraic numbers, then the sets definable in {R, -b, •) 
are the (algebraic) semialgebraic sets. Each of <, — , 0 and 1 are definable: 

X = 1 ■t^'iy[xy = y\, x < y ■t^3z[z^{y — x) = 1\ . 

— The sets definable in (M, -b, •, (r)^^*) are the (real) semialgebraic sets. 

~ The sets definable in (R, -b, e®) are the subexponential sets. (Multiplication 
is definable from addition and exponentiation: For a, &, c G R, we have ab = c 
if and only if there exist x,y,zGM. such that e® = a, e'^ = b, = c, and 
x + y = z.) 

— A set is finitely subanalytic if and only if it is definable in (R, -b, •, (/) ), 
where / ranges over all real-analytic functions / : [—1, 1]” ^ R, n ranging 
over N; see [14]. 

All of the examples given so far have the property that they have some 
explicit, fairly simple, top-down form arising visibly from some nice collection of 
primitives; it’s probably fair to say that this is the exception rather than the rule. 
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For example, the structure (K, +,-,Z) is extremely complicated, involving even 
set-theoretic independence issues; see, for example, Ch. V of [3]. So although the 
generating primitives are familiar, fairly simple (at least, seemingly so) objects, 
the descriptions of the definable sets become increasingly complicated logically. 

The central issue in definability theory is the attempt to understand the 
definable sets generated from given sets of primitives. 



2.3 O-minimality 

Let (M, <) be a dense linearly ordered set without endpoints, and let fOl be an 
expansion of (M, <). Note that every finite union of points and open intervals 
(with endpoints from M U {±oo}) contained in M is parametrically definable in 

The structure fOl is o-minimal (short for order-minimal) if every paramet- 
rically definable subset of M is a finite union of points and open intervals. For 
M = R, this is the same as saying that every parametrically definable subset of 
R has finitely many connected components. For expansions of (R, <, -I-, 1), this 
is the same as requiring only that every definable subset of R have finitely many 
connected components;^ see [8] for a proof. 

Clearly, o-minimality is preserved downward: If (M, <) C dJl' C Tt and Tt is 
o-minimal, then so is TV . 



Examples 

— It’s easy to check that a semilinear subset of Q is a finite union of points 
and intervals, so (Q, <,-l-) is o-minimal. On the other hand, (Q, <,-|-,-) is 
not o-minimal: The definable set {a;GQ:a;^<2}is not a finite union of 
points and open intervals (with rational endpoints). It’s not known if there 
are any proper — that is, strictly larger — o-minimal expansions of (Q, <, -k). 

— If i? is a real-closed field, then {R, -k, • ) is o-minimal, since a semialgebraic 
subset of i? is a finite union of points and open intervals. 

— The structure on R consisting of all finitely subanalytic sets is o-minimal; 
see [14]. 

— Let T be an o-minimal expansion of (R, <,-k). Then the expansion of T 
by exponentiation is o-minimal (hence so is the expansion of T by multi- 
plication); see Y. Peterzil et al. [10] and P. Speissegger [11].) In particular, 
(R, -k, e®) is o-minimal. 

As a counterpoint to the last item above, o-minimal expansions of (R, <,-k) 
that do not define multiplication are exceptional (in the sense that they have 
rather special properties) as are o-minimal expansions of (R, -k, •) that do not 
define exponentiation; see [9] for information. 

O-minimal structures have so many nice properties that it takes pages to 
describe (let alone prove) them; we only touch on the subject here. 

® This resolves some of the “definable versus parametrically definable” problems in 
some earlier papers on o-minimal hybrid systems. 
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Proposition 1 (Uniform Finiteness). Let fOl = (M, <,...) be o-minimal 
and A C be parametrically definable. Then there exists TV G N such that 

for all X € M™, if is finite, then A^ contains less than N elements. 



Proposition 2 (Definable Choice). Let dJl be an o-minimal expansion of a 
dense linearly ordered group (M, <,+,0,l) with a distiguished element 1 > 0. 
Let % ^ A C be definable. Then there is a definable map f : 7t(A) ^ M” 

such that (x, f(x)) € A for all x G 7t(A). 

3 Main Results 

Let H = (Hp)p^p be a family of hybrid systems on M, with parametrized 
flow (j), where the manifold M is assumed to be contained in some cartesian 
product Let 9\{H) denote the structure ( M, F, </>, ) . Throughout this 

section, “definable” means “ definable in . 

Proposition 3. Reach(iL) is definable, as is the set Reach*(iL), consisting of 
all pairs of states (x, y) for which there exists p G P such that y is reachable 
from X in Hp. 

Proof. This is immediate since: (a) the parametrized flow is definable; (b) there 
exists N G N such that for all (p,x,y) G Reach(iL), y is reachable from x in 
Hp via a (hybrid) trajectory of length < N (c) the state spaces are uniformly 
definable, and (d) Reach* (iL) is the projection of Reach(iL) on the last two 
variables. □ 



Corollary 1. Lf there is an algorithm for deciding membership questions for 
sets definable in iR{H), then the reachability problem for H is decidable. 

The first-order theory of real-closed fields (in the language of ordered rings) 
is decidable [13]; hence: 

Corollary 2. Lf L, P and (j> are definable in (M, -I-, •), then the reachability prob- 
lem for H is decidable. 

Proposition 4 (Parameter Selection). Lf iR{H) is o-minimal, then there is 
a definable map T : Reach* (iL) — > P such that for all (x,y) G Reach* (iL), y is 
reachable from x in 

Proof. Apply Definable Choice. □ 

As mentioned in Section 1, we may include (parametrized) initial and final 
sets Co, Cf in families of hybrid systems, and study reachability between them. 
Note that the set of all (p,x,y) such that x G (Co)p, y G {Cf)p, and y is 
reachable from x in Hp is deflnable in {iR{H),Co,C f), the expansion of 9d(iL) 
by Co and Cf. 
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Bisimulations and o-minimality. Suppose that EH(i/) is equipped with initial 
and final states, and is o-minimal. Then, for each p G P, each structure 9d(i/p) 
is also o-minimal. By Theorem 5.3 of [4], each system Hp admits a finite bisim- 
ulation. But more is true: An examination of the proof — together with Uniform 
Finiteness — shows that the bisimulations associated to the Hp are obtained uni- 
formly in p (including a uniform finite bound on the number of iterations needed 
in the bisimulation algorithm). As of this writing, we do not know of any ap- 
plications of this observation that are not obtained more directly by standard 
o-minimal arguments, so we do not give the precise (rather technical) statement 
here. Further investigation of possible applications is in order. 



Time- Abstraction and o-minimality. The time-abstract view of hybrid sys- 
tems ignores all analytic-geometric properties of the continuous part of the hy- 
brid trajectories, but some of the most powerful and striking applications of 
o-minimality are to the study of these properties; see e.g. [17]. 

4 Applications 

We illustrate the results of the previous section by some examples, obtained by 
uniformizing some results from [5] . 

Throughout this section, “semialgebraic” means “definable in (R, -I-,-)” (ar- 
bitrary real constants not included) . 

Consider the parametrized family of hybrid systems {Hp)p^p defined as fol- 
lows. Let U be the set of m-tuples of k x k nilpotent matrices with real entries 
{a k X k matrix A is nilpotent if = 0). Identify U with a semialgebraic subset 
of (R^^)™. Put H = {M, S, I, r, F) where each object is specified below. 

— S = {(x, (1, 2, . . . '■ X € U, 1 < i,j < m\ where V is any semialge- 

braic subset of U. In particular, we use a fixed set of locations (1,2,... , m) 
and P = V X {(1, 2, . . . , m)}. 

— M = R'= 

— I = P X M'". That is, I is identified with a subset of 

— T is a semialgebraic set satisfying the conditions of Definition 1 . 

— For each p = (an(l),... ,Ofcfc(l),... ,an(m),... , afcfc(m), (1, 2, . . . ,m)) € 
P and X G M define F{p, x) = (A(l)x, . . . , A{m)x). So, each component of 
F(p, •) is a linear vector field with a nilpotent matrix. 

Since the flows of the vector fields defined above consist of polynomial func- 
tions in t with semialgebraic coefficients, such flows are themselves semialgebraic. 

By Proposition 3, there is a semialgebraic function <1> such that for any choice 
of nilpotent matrices Ai,... , Am satisfying (Ai,... ,A^) G V and any two 
points a;, y in {1, . . . , m} x R^, y is reachable from x in the corresponding hybrid 
system if and only if <?(p, x,y) = 0 (where p = (an(l), . . . , Ofcfc(l), . . . , an(m), 
• ,Ofefe(m),(l,2, ... ,m))). 

As another example, consider a finite set T> of diagonal matrices with rational 
entries and let A be a semialgebraic subset of GLfe(R). Let all data be as in 
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the previous example except that we replace V with the set of (diagonalizable) 
matrices of the form TAT~^, where T & A and A G P. In this case Reach(_ff) is 
definable in (M, +, •, exp). Moreover, it was shown in [5] that Reach(i/) is in fact 
semialgebraic. Also, a kind of converse is true: If Reach(i/) is semialgebraic, and 
the matrices involved are diagonalizable, then, up to scaling, the set T> must be 
finite. Using results from [6,7], this is not difficult to prove, but it would take us 
too far afield to do it here. 

The set Reach(i7) may be definable in an o-minimal structure even if the 
parametrized flow is not. Consider a system as above but where the matrices 
are similar to one with purely imaginary eigenvalues (with rational imaginary 
part) and of a special real Jordan form (having 2x2 blocks). Then the flows are 
complete and periodic and so the system is not o-minimal. On the other hand, 
using the calculations in §5.3 of [5] one can show that Reach(i7) is, in fact, 
semialgebraic. 



Conclusion 

In this paper, we introduced parametrized families of hybrid systems and ob- 
tained some reachability results uniformly in parameters. 

While some decidability results are available for special classes of systems, 
they all rely on the decidability of the theory of real closed fields. New interesting 
results may be obtained by finding classes of formulas decidable within the theory 
of the field of reals with exponentiation. 
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Abstract. This paper studies the existence of solutions to a class of 
hybrid automata in which the underlying continuous dynamics are rep- 
resented by inhomogeneous linear time-invariant systems whose inputs 
are controls that can be determined by the user. The principal result of 
the paper is a procedure that searches for global periodic non-terminating 
solutions of systems having a single cycle. 



1 Introduction 

A controlled hybrid automaton is a hybrid automaton [Alu93] [Lyn96] whose un- 
derlying continuous-state dynamics are modeled as inhomogeneous differential 
equations. In particular, we restrict our attention to continuous-dynamics repre- 
sented by linear time-invariant (LTI) systems of the form x{t) = Ax{t) + Bu{t) 
where A G 3?”^”, x(t') € 3?”, B G 3?”, and u(t) G 3?. The scalar u(t) is the con- 
trol input at time t G 3? and it is selected by the system designer. In this paper, 
we further restrict our attention to systems with only a single cycle. This paper 
presents preliminary work examining conditions under which non-chattering and 
non-terminating solutions exist for the controlled hybrid automaton. The prin- 
cipal result is a gradient-following algorithm that provides a systematic means 
of searching for global periodic non-terminating solutions of systems with single 
cycles. 

The remainder of the paper is organized as follows. Section 2 defines the 
controlled hybrid automaton and defines the sense in which a hybrid trajectory 
satisfies such a system. Section 3 outlines conditions for the existence of local 
non-chattering solutions. Section 4 outlines conditions for global periodic non- 
terminating system trajectories. Final remarks are found in section 5. 



2 Controlled Hybrid Automata 

A controlled hybrid automaton is a labeled digraph characterized by the 4-tuple 
(N,A,£n,^a)- Af is a set of nodes in the directed graph (represented graphically 
as open circles). The set of nodes is usually taken as a subset of the positive 
integers. A C A x is a set of directed arcs between nodes. The arc (z, j) from 
node i to node j is graphically represented as an arrow that starts at node i and 
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terminates at node j. The ordered pair {N, A) is the finite automaton associated 
with the hybrid system. The map £n ■ N ~x 3?”^" x 3?" associates a pair of 
real vectors with the node. In particular, the label t'Ar(i) = associates a 

real matrix Ai G 3?"^" and a real matrix (vector) Bi G 3?" with the tth node. 
Associated with node i is the following inhomogeneous differential equation, 

x(t) = Aix{t) + Biu{t) (1) 

where x{t) G 3?" and u{t) G 3?. Equation 1 is called the modal equation of the 
tth node. The map £a ■ A — > 7^(3?”) maps an arc ai G A onto a collection of 
vectors in 3?". In particular, if arc ai is labeled as 

£A(ai) = {vn,vi2, ■ ■ ■ ,vipi} 

then we can associate with a\ a special subset T(ai) C 3?” that is called the 
guard of the arc. The guard is defined to be the convex hull of the points in the 
collection t'^(ai). By the standard representation theorems for convex sets, we 
therefore know that T(ai) can be characterized as 

{ Pi Pi 

X = : ^Aii = l , Ah > 0 , vu G £A{ai) 

i=l i=l 

From the above equation it should be clear that the vectors in £A{ai) are the 
extreme points (vertices) for convex polytope T(oi). 

Remark: In this paper we’ve adopted the convention of representing guards 
as convex combinations of vertices, rather than as feasible regions bounded by 
hypersurfaces. 

A controlled hybrid trajectory z : iR ^ X x N xU is a, function mapping a real 
number r G 3? onto the ordered triple (x(r), ^(t), m(t)) where x{t) G A C 3?" 
is called the the continuous state, fr) G N is called the discrete state, and 
u{t) G C/ C 3? is the control. It is assumed that A is a closed connected subset 
of 3?” and it is assumed that U is a compact subset of 3?. 

A time instant r G 3? is said to be regular if z is continuous at t. ( In 
this case, we assume that N is equipped with a discrete metric d{i,j) = 1 if 
i f j and is zero ifi = j). If r is not a regular point, then it is called a switching 
instant. Controlled hybrid trajectories with a finite number of switching instants 
in any closed time interval are said to be non- chattering. A controlled trajectory 
with an infinite number of switching instants is said to be non-terminating. The 
trajectory is said to be local if its maximum interval of existence has the form 
['Ha, To + T) and T is finite. The trajectory is said to be global if its maximum 
interval of existence of [tq, oo). 

A controlled hybrid trajectory z : [tq, tq-GT) ^ X x N x U is said to satisfy 
the controlled hybrid automaton {N,A,£ai,£a) with initial condition xq G X 
and zq G A at time tq G 3? if and only if 
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— a;(ro) = a;o, z(to) = io, and m(to) G U. 

— For all closed intervals [ra,Th] containing no switching instant, there exists 
a, j G N, an absolutely continuous [Aub84] trajectory x : [Ta,Tb] X, 
and a measurable control u : [Ta,Tb] U such that i(r) = j and x(t) = 
Ajx(r) + Bju{r) for all r G [Ta,Tb]. 

— At any switching instant, Tg G 1ft, there exists a j and k in N such that 
(j, k) G A, lini^^^- i(r) = k, and x{ts) G F((j, k)). 

Such trajectories are also said to be solutions of the hybrid automaton. A system 
that can generate non-chattering solutions will be said to be non-Zeno. A system 
that can generate non-terminating solutions will be said to be deadlock-free. 
Remark: Note that switching can occur anywhere within the guard set. 
Consider a controlled trajectory 2 defined over [tq, tq-I-T) with discrete state 
trajectory i : [to,to -\-T) ^ N. The sequence of discrete states associated with i 
can be denoted by the string cr G N* where N* is the Kleene closure of N. We 
refer to cr as the trajectory’s event sequence. By the pumping lemma [Dav83] , we 
know any finite length event sequence can be decomposed as cr = usv such that 
the event sequence us"u (for any positive n) is accepted by the finite automaton 
{N, A) associated with our system. This means that the sequence s represents 
a cycle of events. If there exist trajectories such that the hybrid automaton can 
execute this cycle repeatedly, then we say that the hybrid automaton is deadlock- 
free with respect to s. A key issue in the study of hybrid automata (whether 
or not they are controlled) concerns the deadlock-freedom of such systems. This 
issue is, in essence, a question concerns the existence of global non-terminating 
solutions to hybrid automata. 

3 Local Non-chattering Solntions 

Figure 1 shows a cyclic controlled hybrid automaton. Assume that the initial 
continuous state at time tq is xq and that the initial discrete state is to = 1. In 
this section, we briefly examine conditions ensuring the existence of a T > 0 such 
that there exists a controlled hybrid trajectory 2 over the interval [to,to -I- T) 
that is a solution to the controlled hybrid automaton. In this section, we consider 
two distinct cases. The first case occurs when xg is not in T((I, 2)). The second 
case occurs when xg G T((I,2)). 

The following results are a routine application of viability theory [Aub84] 
and are presented here for the sake of completeness. See [Aub84] for a precise 
statement of the definitions and theorems cited below. 

Let’s assume that xq ^ A((l, 2)). Since the guards are closed sets, this means 
that xg belongs to an open set so we can enclose xg in an open neighborhood 
Bg{xg) that is contained completely within the complement of the two guards. 
Over this neighborhood we can define a set valued mapping F’ : A — > 7^(3ft”) 
that takes the value 



F{x) = {Aix -\- B\u , u G U} 



(2) 
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r(l,2)={Vii , Vi2 , ... ,Vipi} 




Fig. 1. cyclic controlled hybrid automata 



at X. Since U is compact, we know that F(x) will be an upper semi-continuous 
(use) convex-valued map and we can use the USC Existence Theorem to infer 
that there exists a 5 > 0 and an absolutely continuous x : [to,to -I- <5) ^ X 
that satisfies the differential inclusion x S F{x). By the Measurable Selection 
Theorem we can infer the existence of a measurable u over the same interval. 
Since there are no switching instants over this time interval, we know that i(r) = 
1 for all r G [tq, tq -I- (5). This particular case, therefore, has a hybrid trajectory 
satisfying the system. 

The other major case of interest occurs when xo G T((l, 2)). Let’s first con- 
sider the case when xq G ^((1,2)) and xq ^ T((2,l)). We consider the set 
valued map, F{x), of equation 2. Since xq may lie on the boundary of T((l,2)), 
we cannot enclose xq in an open neighborhood over which F is defined. How- 
ever, F is upper semicontinuous and provided we can ensure that F satisfies the 
tangential condition [Aub84], then we can use the Viability Theorem to ensure 
the existence of of an absolutely continuous solution to the differential inclusion 
X G F{x) that is viable in T((l,2)). Finally, the Measurable Selection Theorem 
ensures the existence of the desired Lebesgue measurable control, u. 

Now let’s consider the case when x G K = T((l,2)) n F{{2, 1)) Consider a 
set valued map, F, that takes the value 

F{x) = {AiX + BiU : rt G U, i = {1, 2}} 

at point Xq G K. Since K is compact, then xq may lie on the boundary of K and 
cannot be enclosed in an open neighborhood over which F is defined. Morever, 
F may not be upper semicontinuous over K. Therefore we cannot use the USC 
Existence Theorem to establish the existence of local trajectories. However, the 
convex hull m(F(x)) of F(x) is clearly convex valued and Lipschitzean on K. 
Moreover, if we can ensure that the tangential condition holds, then the Vi- 
ability Theorem ensures the existence of an absolutely continuous solution to 
X G co(F(x)) that is viable in K. The Relaxation Theorem can then be used 
to infer the existence of absolutely continuous solutions to the original differen- 
tial inclusion x G F(x). As before, an application of the Measurable Selection 
Theorem ensures the existence of a Lebesgue measurable u for this selected tra- 
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jectory x. Finally, since x is absolutely continuous, we know that each closed 
interval has a finite number of switching instants in which the discrete state 
changes value thereby establishing that the trajectory is non-chattering. We’ve 
therefore established the existence of a local non-chattering solution provided 
the tangential condition found in the viability thoerem is satisfied. 

4 Global Nonterminating Solutions 

A global nonterminating solution to a controlled hybrid automaton is a hy- 
brid trajectory that exists over [to,oo) and that generates an infinite number of 
switching instants. This section studies the existence of global non-terminating 
periodic trajectories for the cyclic controlled hybrid automaton in figure 1. This 
hybrid automaton consists of two nodes (1 and 2) and two arcs. The fth node 
is labeled with the system (Ai,Bi) and arcs (1,2) and (2,1) are labeled with 
vertex collections Vi = {un, ui 2 , • ’ ’ > I’lpi } and V 2 = {u2i, ^ 22 , • ’ ’ > re- 

spectively. The ith guard associated with the arc entering the fth node denoted 
as Fi = cb(Vj). 

Consider one of the modal systems (f = 1 or 2) 

x{t) = Aix{t) + Biu{t) (3) 

where x{t) G 3?”, Ai G Bi G 3?”, and u{t) G 3?. We say a state u G 3?” 

is reachable from a state w G 3?" if there exists a time T > 0 and a measurable 
control u : [tq, tq+T] — > U such that the controlled trajectory x : [to,tq+T] X 
satisfies equation 3 with x{tq) = w and x{tq + T) = v. The set of all points from 
which V is reachable is called the preset of v and will be denoted as pre(u). The 
preset of a subset F C X is denoted as pre(T) and is defined by the equation 
pre(r) = U„6r pre(u). A necessary and sufficient condition [Ant97] for w to lie 
in the preset of v is that there exist a T > 0 such that 

— V € TZ(Ci) (4) 

where 

Ci= [B,AB,AfB,---A^~^B,] (5) 

is called the controllability matrix for the fth modal system. The range space of 

Ci is denoted as TZ{Ci) and we assume it has a dimension of r^. In the following 
discussion, is a matrix of dimension n x ri {i = 1,2) whose columns are 
standard basis vectors for the subspace TZ{Ci). 

Remark: Note that this reachability condition applies when the control u 
can be unbounded (as is the case in so-called impulsive controls). 

Remark: Note that the term reachability is used in a somewhat different 
sense that what is found in traditional algorithmic verification [Alu95]. Tradi- 
tional hybrid automata have homoegeneous modal equations and as a result 
pre(u) for a fixed transition time T consists of a single point. In view of equation 
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4, it is apparent that the introduction of the control extends the preset of v to 
a set formed from affine varieties of the controllability subspace. 

We assume an initial condition xq G X and ig = 1. The question to be 
answered is whether there exists a control u and a pair of switching times Ti 
and T 2 such that a hybrid trajectory 2 ; is a solution of the system over the interval 
[tq, 00 ) and such that z generates an infinite sequence of switching instants 



'Ha, Til, T21, T21, T22, • • • J Tij, ■ ■ ■ 



where is the jth switching instant out of mode i, T 2 j — ry = Ti, and Tij+i — 
T 2 j = T 2 . In other words, the hybrid trajectory z is nonterminating and periodic 
in time. 

By our definition of a solution to a controlled hybrid automaton, we know 
that the continuous state at each switching instant must lie in the appropriate 
guard set. In other words x{Tij) € Fi for all j and i = 1,2. Since the guards 
are convex polytopes, the switching instants x{Tij) can be represented as convex 
combination of the form 



Pi 

x{Tlj) = ^ XuVu 

i=l 

P2 

x(T2j) = ^ \2%V2i 
i=l 

where Ay > 0 for i = 1,2 and all j and where ^ 

Therefore if we are to have a nonterminating behavior, we know that x{t 2 j) 
must be reachable from x(Ty ) in time Ti and a;(rij-|_i) is reachable from x{T 2 j) 
in time T 2 . From equation 4, this condition is satisfied if there exist vectors 
/?1 = [/3ii,/3i2,---,/3iri]^ and /?2 = [^ 21 , /?22, • ’ ’ , ^ 2 r 2 ]^ such that 



ri Pi P2 

0 = ^ Pueu + Xuvii - \2iV2i ( 6 ) 

i — 1 2=1 2=1 

V2 Pi P2 

0 = ^ I32te2t - XiiVu + ^ \2iV2z (7) 

2=1 2=1 2=1 

Pi 

1 = ^Aii (8) 

i=l 

0 > Aii , (i = 1, . . . ,pi) (9) 

P2 

1 = E^2i (10) 

2=1 

0 > A2i = I,. . . ,P2) (11) 



We reframe equations 6, 7, 8, and 10 as the matrix vector equation 



c = Sr] 



( 12 ) 
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02nxl 

1 

1 



El 0j^xr2 


e^iT’iVi -V 2 






Onxri E2 


-Vi e^=^=^V2 




^2 


Olxri 0ixr2 


llXpi ^lXp2 




Ai 


Olxri 0ixr2 


^IXpi llXp2 




.A 2 



y 



(13) 

(14) 



where Vi = [un, Ui 2 , • • • , I’lpJ, V 2 = [^ 21 ,^ 22 , • • • , V 2 P 2 ] (matrices whose columns 
are the guard vertices) , c = [0„xi | l 2 xi]^, V = W' , z = [A, • ’ ’ , 
and y = [Af, A^]^. 

Remark: The vectors Ai, A 2 , /?i, and /?2 satisfying equations 6 to 11 char- 
acterize affine spaces which are mutually reachable from each other. Note that 
these solutions provide an explicit characterization of mutually reachable pre- 
sets in terms of the vertices of the guards. This explicit representation of the 
presets of the system is the reason why the guards were represented as convex 
combinations of vertices. 

By the theorem of the alternative [Baz93], a necessary and sufficient condition 
for equations 6 to 1 1 to have a non-negative solution is that there exist no vector 
X such that 



Gx < 0 , Fx = 0 , c^x > 0 (15) 

The solution to equation 15 can be checked by solving the associated linear 
program 

maximize : c^x 
subject to : Gx < 0 

Fx = 0 (16) 

Solutions to the above problem have a special form due to the fact that Gx 
forms a polytopic cone whose apex is at the origin. Figure 2 shows the possi- 
ble situations that can occur with this linear program. The figure shows that 
solutions to this linear program are either unbounded and positive or bounded 
and equal to zero. If the solution is x = 0, then the alternative problem in 
equation 15 has no solution since c^x = 0. This means that equation 12 has a 
non-negative solution and we can infer that for the fixed time T that the guard 
r is reachable from cb(W). If an unbounded solution occurs then equation 12 
has no non-negative solutions and we can infer that for the given T , the guard 
r is not directly reachable from cb(W). 

Remark: A feasible solution at x = 0 implies that the specified cycle exists 
between the two guard sets and an unbounded solution implies that a cycle does 
not exist with the specified transition times T\ and T 2 . Note that the existence of 
an unbounded solution does not imply that the guards don’t support a recurrent 
cycle, for there may be other transition times for which the cycle exists and it 
may be possible that the guards support a cycle in which the transitions are not 
necessarily periodic. 
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zero feasible solution unbounded solution 

The alternative problem’s linear program must have a solution 
lying in the null space of F will either be unbounded or lie at 
the apex of the cone formed by the equation Gx < 0. 

Fig. 2. The Alternative Problem’s Linear Program 



If the linear program in equation 16 returns an unbounded solution, then 
it may be possible to adjust the transition times T\ and T 2 to force a solution 
at X = 0. From duality theory [Baz93], we know that if the primal problem in 
equation 16 is unbounded, then its dual is infeasible. The infeasibility of the dual 
can be readily checked by examining the Lagrange multipliers associated with 
the inequality constraints of the primal problem. These Lagrange multipliers are 
generated by any primal-dual linear programming algorithm. They represent 
part of the solution to the dual problem and are used to help assess how close 
a linear programming algorithm is to being finished. If these multipliers are 
negative, then the dual is infeasible and we can immediately conclude that the 
guard does not support a cycle at the specified transition times. 

The preceding observation suggests a simple heuristic method for adjusting 
the times T\ and T 2 in order to force the dual problem to be feasible. Let Vk de- 
note the fcth Lagrange multiplier associated with the linear program’s inequality 
constraints. We define a performance measure associated with a specific pair of 
times (Ti,T 2 ) as 



-^((T’i,T 2)) = minfcj/fc (17) 

This measure identifies the smallest Lagrange multiplier and uses it as a measure 
of how close the dual problem is to being feasible. The obvious strategy is to 
perturb the current transition times T\ and T 2 , observe the change in J and then 
select a new set of times that will increase J. We continue in this manner until 
J becomes positive. 

This idea was tested using the following, very simple, search strategy. First 
initialize the search by selecting a set of times T\ and T 2 . The search is then 
executed by the following steps. 



On the Existence of Solutions to Controlled Hybrid Automata 237 



1. Perturb (Ti,T 2 ) by a small adjustment (5 > 0 and solve the linear program 
(equation 16) for points {Ti,T 2 ), (T’i,T '2 + i^)) and (Ti + 5,T2), ■ 

2. If any of these linear programs are feasible, then the system supports a 
periodic solution and we’re finished. 

3. If all of these linear programs are infeasible, then the Lagrange multipliers 
for each problem are used to compute costs J((Ti,T 2 )), J{{Ti + 15, T 2 )), and 

j((Ti,r2 + ^)). 

4 . Select a new set of times, (Ti, Tjj) according to the following rule, 

r Ti + ,5, if J((Ti, T2)) < J((Ti + ,5, T2)) 

T[ =)Ti-6,H J((Ti, T2)) > J((Ti + ,5, T 2 )) 

[ Ti , otherwise 

rr2 + ^, if J((Ti,T2)) < J((Ti,T2 + <5)) 

T^=<T2- 6, if J(Ti, T2)) > J((Ti, T2 + <5)) 

[ T 2 , otherwise 

5. Set Ti = T{, T 2 = T 2 and return to step 1. 

What this algorithm does is attempt to solve a nonlinear optimization problem 
using a gradient-following strategy. The preceding steps describe the master 
algorithm that uses the results of the linear program in equation 16 to select a 
set of better times. 

Remark: In the procedure we’ve chosen, of course, there are no guarantees 
that this search will terminate as it is currently unclear how the times, T\ and 
T 2 are related to the problem’s Lagrange multipliers. Nonetheless, this search 
program provides what seems to be a very pragmatic method for testing for the 
existence of global solutions and if it does terminate, then we know for certain 
that the cycle is live. 

The following example illustrates the proposed search algorithm. Consider 
the cyclic hybrid automaton shown in figure 1 where the nodes are labeled as 



f'Ar(l) 

£at(2) 



0 4 4 

I/40J ’ [1 

0 -10 
- 1/10 0 



10 

1 



The arcs are labeled with vertex collections 




A MatLab script was written to implement the master program given above and 
this script was used to search for a global non-terminating solution of our hybrid 
automaton. The lefthand plot in figure 3 illustrates the results of this search. 
The a;-axis shows the times Ti and T 2 whereas the y-axis shows the value of the 
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Minimum-energy trajectory 




Fig. 3. Performance Measure J versus Switching Intervals and Simulation Re- 
sults 



cost J{{Ti,T 2 )). The search starts with Ti = T 2 = 1. This starting point is in 
the middle of the at-axis. The intermediate values of J computed by the master 
algorithm are shown by the solid and dotted line trajectories (the dotted line 
for Ti and the solid line for T 2 ). We see that the master algorithm computes 
a monotone sequence of times in which T 2 is decreasing and T\ is increasing. 
After a finite number of iterations the master program has identified the times 
T 2 = 0.29 and Ti = 1.7 as points whose linear programs have non-negative 
Lagrange multipliers. These points, therefore, are feasible and characterize a 
global non-terminating periodic cycle for this system. 

The master algorithm allows us to assert that a global periodic solution to 
this system exists. The intermediate results of the algorithm also allow us to 
characterize the switching sets and we can actually identify some of the control 
strategies, u, that enforce this periodic solution. This additional information is 
contained in all non-negative solution vectors Ai and A 2 satisfying our system 
Srj = c. The set of all solutions can be parameterized as 77 G fjpQ -I- null(S) where 
r]pO is a particular solution to the inhomogeneous equation c = S 77 . Note that 
this implies that the mutually reachable sets in the guards are affine sets. It 
was our parametrization of the guard as a convex combination of vertices that 
allowed us to obtain such a simple and explicit representation of these sets. For 
the example above, we can readily identify these sets in which the particular 
solution is rJpQ = [O —1.94 1.39 —1.10 0.71 0 0.048 0.95]”^ and the null space of 
S is spanned by the columns of the matrix 



-0.9237 0 

0.1380 -0.4162 
0.1820 -0.3445 
-0.2321 0.1920 
0.0501 0.1525 
-0.1536 -0.6431 
0.1144 0.4389 
0.0393 0.2042 
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Let’s now look at the controls required to enforce the nonterminating cycle. 
We first look at a pair of specific switching points in the guards and then identify 
an open loop control enforcing a periodic trajectory between these points. One 
specific set of switching points for our example system is 

x{ti) = ViAi = [-2.6569 0.6569] 
x{t2) = V 2 A 2 = [3 0.9903] 

The existence of a control driving the system between these two points is guaran- 
teed by the termination of our master program. What is this open loop control? 
We have many choices and one obvious choice is the minimum energy control 
strategy. The minimum energy control u(t) that transfers the first modal system 
from the initial state x{t\) to target state x{t 2 ) satisfies the condition 

x{t2) - G Tl{Ci) 



is given by 

ui{t) = 

where 771 is the solution of the equation 

TTi(0,ri)77i =x{t 2) -e^^'^^xin) 

Wi(0, Ti) is the controllability Gramian of (Ai, Bi). For the system at hand the 
solution is 

u\{t) = -ipf; = 0.3349e“‘ 

(.2T2 _ I 

Similarly the minimum energy solution for the second mode is 

y- 2 {t) = = -0.9758e* 

1 — e ^-^2 

The hybrid automaton’s trajectory with this minimum-energy control is shown in 
the righthand plot in figure 3. This figure shows the state space for our system. 
The two triangular regions in this plot represent the guards. For the specific 
choice of points x{t\) and x{t2), we use the control u{t) identified above to 
compute the state trajectory between these points. The solid line in figure 3 
shows the resulting controlled trajectory. 

It is, of course, possible to obtain other controls realizing this cycle. For in- 
stance, an “impulsive” control strategy can be employed, in which we impulsively 
drive the system state along an affine variety of the controllable subspace and 
then allow the system to relax into the guard. (In other words we let u(t) be an 
impulse function of specified magnitude) . The lefthand plot of figure 4 illustrates 
the state trajectory generated by this control law. As in figure 3, we are looking 
at the system’s state space. The triangular regions represent the guards and the 
solid lines denote the state trajectory generated by the impulsive control. 
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Irrjisive trajectory 





Fig. 4. Impulsive simulation results and sweeping out the viability kernel 



For a specific pair of times, our master algorithm identifies all states in the 
guards that are mutually reachable from each other. If we were able to iden- 
tify a range of feasible switching times, then it should be possible to identify 
larger subsets of the guard that are mutually reachable from each other. The 
complete set of states in the guards that are mutually reachable from each other 
(under any control strategy) is sometimes referred to as the viability kernel. Our 
algorithm, therefore, provides a means of approximating the viability kernel. 
Note that this is an under approximation to the viable set (as opposed to the 
over-approximation computed by model checking algorithms [Alu95]). For our 
specific example, we were able to identify a range of times over which periodic 
solutions could be guaranteed. This range was computed to be. 1.7 < Ti < 3.75 
and .1059 < T 2 < .29. The set of points swept out by these various times is 
shown in the righthand plot of figure 4. We’ve compared this set to the actual 
viability kernel for this system and the specified bounds appear to provide a 
close approximation to the actual viability kernel. 

Remark: The failure of the master program to find any feasible solution 
does not guarantee that a global solution doesn’t exist. How quickly we find a 
feasible solution clearly depends upon the type of search strategy the master 
program uses and depends on our initial guess. 

Remark: Our approach focuses on identifying periodic global solutions and 
obviously it may be possible that this is overly restrictive. For instance, it may 
be possible that only chaotic trajectories exist between the two guards, or that 
a more complex periodic behavior exists between the two guards. 

Remark: The preceding discussion focused on establishing non-terminating 
solutions to a rather simple hybrid automaton. This problem was chosen as a 
canonical problem in the sense that its solution may provide a foundation upon 
which to establish the existence of global solutions to more complex systems. How 
might this be done? This is the topic of another paper, but we can speculate on a 
possible strategy based on prior results on the role of cycles in hybrid automata 
[He98] [Zhi98]. Essentially, the argument runs as follows. From the pumping 



On the Existence of Solutions to Controlled Hybrid Automata 241 



lemma, we know that the logical behavior generated by any automaton can be 
broken down into a concatenation oi fundamental cycles. This paper, essentially, 
is proposing a pragmatic way for determining whether a given fundamental cy- 
cle is viable. Moreover, our algorithm computes an under-approximation to the 
cycle’s viability kernel that can be very good (as shown in our example). Let’s as- 
sume we can determine controls guaranteeing all fundamental cycles are viable. 
Given a specific concatenation of cycles in the system, we then look at the inter- 
section of viable sets of contiguous cycles (actually look at the approximations 
computed using the methods in this paper). If this intersection is non-empty, 
it should be possible to determine control strategies enforcing the viability of 
arbitrary concatenations of fundamental cycles and thereby ensure the viability 
of the entire complex system. As noted above, whether or not this approach will 
work is still under study. 



5 Conclusions 

Controlled hybrid automata are automata in which a user-determined input con- 
trol signal can be used to help supervise overall system behavior. In this paper, 
we assumed the modal systems were linear and time invariant with polytopic 
guards formed from the convex combination of vertices. This paper studied the 
existence of solutions to this class of hybrid system. A routine application of 
Viability theory was used to characterize the existence of local trajectories. This 
paper presented a necessary and sufficient condition for the existence of a global 
periodic non-terminating trajectory with specified switching intervals. This re- 
sult was used to propose a gradient following search strategy for determining 
a set of switching intervals ensuring a global nonterminating trajectory. The 
proposed method also provides an under-approximation of the cycle’s viability 
kernel that could be used in extending this work to more complex switching 
systems. A distinguishing feature of this study is the explicit use of the open 
loop control signal u(t) as a means of enforcing a cycle’s viability. 

This work is preliminary in that there are still a number of open questions 
that need to be answered. There is uncertainty over the performance of the pro- 
posed search algorithm. It should be noted, however, that such gradient follow- 
ing heuristics often work extremely well on real-life problems, so this approach 
may still be a pragmatic approach to hybrid system verification. Another open 
issue concerns the conservatism imposed by confining our search to periodic 
non-terminating solutions. While this might appear to be very restrictive on the 
surface, it must be realized that the proposed approach can actually identify a 
set of periodic solutions and that other non-periodic solutions might be seen as 
limiting points of this set. Another interesting issue brought up by this paper 
is the explicit use of control. Traditional analyses of hybrid systems assume no 
control and the verification process can be seen as a ’’take it or leave it” analysis 
that provides little guidance on determining how ” close” a system is to being vi- 
able. The use of control advocated in this paper may provide the system designer 
with a more sophisticated approach to verification in which control becomes a 
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necessary component in system design. Finally, this paper has focused on hybrid 
systems containing only one cycle. This simple problem is viewed as a necessary 
starting place for the analysis of more complex hybrid systems and the details 
of this later analysis will be the subject of future papers. 
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Abstract. This paper is concerned with global asymptotic stabilization 
of continuous-time control systems by means of quantized feedback. For 
linear systems, a hybrid control strategy for dealing with this problem 
was recently proposed by Roger Brockett and the author. The solution 
is based on making discrete on-line adjustments to the sensitivity of 
the quantizer. In the present paper we extend this method to a class of 
nonlinear systems. 



1 Introduction 

We study the problem of stabilizing a control system with quantized state feed- 
back. This problem consists in designing a stabilizing control law which, instead 
of using the measurements of the system’s state x directly, is only allowed to 
depend on the quantized measurements q/\{x). Here qa is a piecewise constant 
function with a finite set of values, called a quantizer (see Figure 1). 

* <j4*) 





Fig. 1. Quantized state feedback 



Given a positive real number A and a positive integer M , we denote by T 
the set {fc G Z : —M <k< M}, and define the function g/i : M ^ {kA \ k & 
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by the formula 

(kA if {k — ll2)A<x<{k+l/2)A where fc G I 
qA{x)=\MA li x>{M + l/2)A (1) 

\-MA a x<-{M+l/2)A 

We will call A the sensitivity of the quantizer and M the saturation constant. 
When X G K", the quantized vector qA{x) is defined componentwise using the 
function (1). Geometrically, M” is thereby divided into a finite number of recti- 
linear regions called quantization blocks, each corresponding to a fixed value of 
qA- 

Quantizers of the kind described above are commonly used to model finite- 
precision effects which arise, for example, when the state measurements to be 
used for feedback are processed by a digital computer or transmitted via a dig- 
ital communication channel. In such situations, the sensitivity and the number 
of values of the quantizer are given a priori and cannot be changed by the con- 
trol designer. This places significant constraints on what can be accomplished 
by using feedback. In particular, we see from (1) that qA{x) = 0 for all x in 
a sufficiently small neighborhood of the origin, hence asymptotic stabilization 
is impossible (unless there exists an open-loop control law that drives all the 
states in this neighborhood to the origin). See, e.g., [3,5,7,14,20,21,22] for more 
information regarding quantization issues. 

However, numerical quantization in computer-controlled systems is just one 
example of a situation where the problem of developing techniques for quantized 
feedback stabilization is relevant. There are many other cases in which a function 
of the form (1) can be used to represent information that is available to the 
controller. For instance, imagine having a sensor that determines whether the 
temperature of a certain object is “normal”, “too high”, or “too low”. Such 
a sensor can be modeled by a quantizer with saturation constant M = 1 (of 
course, a higher value of M can also be used if one demands more information 
from the sensor). Since it is reasonable to assume that one is allowed to adjust 
the threshold settings from time to time, the sensitivity A is not necessarily 
fixed in this case. As another example, a video or photographic camera with 
zooming capability can be described by a quantizer of the type considered here, 
with M determined by the number of pixels. By zooming in or zooming out, one 
effectively varies the sensitivity of such a quantizer. 

Following [2], we take the approach that it is possible to change the sensitivity 
A (but not the saturation constant M) of the quantizer on the basis of available 
quantized measurements. The problem then is to design a quantized feedback law 
that yields asymptotic stability of the equilibrium a; = 0. The above discussion 
suggests that this problem, besides being of theoretical interest, is also quite 
meaningful in many practical applications. We will assume that the given system 
evolves in continuous time. The values of A, on the other hand, will belong to a 
discrete set and will be updated at discrete instants of time (these events will be 
triggered by the values of a suitable Lyapunov function) . The closed- loop system 
will therefore be a hybrid system, with continuous state x and discrete state A. 
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Another logical variable 2 (for “zoom”) will also be used. The systems obtained 
in this paper fall into the general framework for hybrid systems presented in 

[1] (see [2] for details). As in [2], all feedback control laws will be constructed 
explicitly, and solutions to all differential equations are well defined, with the 
understanding that they are to be interpreted in the sense of Filippov [8] if 
necessary. 

Control policies based on the above idea of making discrete on-line adjust- 
ments to the sensitivity A of the quantizer will be referred to as hybrid quantized 
feedback control policies. Our reasons for adopting a hybrid control approach, 
rather than varying A continuously, are threefold. First, in specific situations 
there may be some constraints on how many values A is allowed to take and 
how frequently it can be changed. Thus a discrete adjustment policy for A is 
more natural and easier to implement than a continuous one. Second, the anal- 
ysis of hybrid systems obtained in this way is usually much more tractable than 
that of systems resulting from continuous adjustments of Z\. In fact, we will see 
that a method based on computation of invariant regions defined by level sets of 
a Lyapunov function provides a very simple and effective tool for studying the 
behavior of the closed-loop system. Finally, a discrete adjustment policy is more 
robust with respect to time delays, which constitute another important issue to 
consider in the present “limited information feedback” setting (cf. [22]). 

The hybrid control approach to the quantized feedback stabilization problem 
was first introduced by Roger Brockett and the author in the recent paper [2]. 
That paper deals with linear control systems. It is shown there that if a linear 
system can be stabilized by a linear feedback law, then it can also be stabilized 
by a hybrid quantized feedback control policy. One strategy proposed in [2] for 
achieving global asymptotic stability consists of two stages. First, since the initial 
state is unknown, we “zoom out” by increasing A until the state of the system 
can be adequately measured. Second, we “zoom in” by decreasing A in such a 
way as to drive the state to 0. (The discrete “zoom” variable z equals 1 in the 
first case and —1 in the second case.) 

The goal of this paper is to extend the above method to nonlinear systems. 
It can be shown via a linearization argument that by using the approach of 

[2] one can obtain local asymptotic stability for a nonlinear system, provided 
that the corresponding linearized system is stabilizable (see [11]). Here we are 
concerned with the problem of achieving global or at least semi-global asymptotic 
stability. Working with a given nonlinear system directly, one gains an advantage 
even if only local asymptotic stability is sought, because the linearization of a 
stabilizable nonlinear system may fail to be stabilizable. We will demonstrate 
that the techniques developed in [2] can be extended in a natural way to those 
nonlinear systems that are “externally stabilizable” in a certain sense to be made 
precise below. For linear systems, external stabilizability follows automatically 
from the usual (internal) stabilizability, but for nonlinear systems this leads to 
a nontrivial problem which is a subject of ongoing research. We thus reveal an 
interesting interplay between the problem of quantized feedback stabilization, 
the theory of hybrid systems, and recent advances in nonlinear control design. 
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The rest of the paper is structured as follows. In Section 2 we review a 
result from [2] concerning linear systems. We provide its complete proof (slightly 
modified from the original version). This method is then applied to a class of 
nonlinear systems in Section 3, where the main results of this paper are obtained. 
The ideas behind the control strategy are essentially the same as in the linear 
case; however, the analysis is inherently nonlinear and involves several concepts 
from the modern nonlinear control literature. Section 4 discusses these concepts 
and relevant developments, as well as the results presented here and their possible 
extensions. 



2 Linear Systems 



Given a quantizer q^, by its saturation region we will mean the union of those 
quantization blocks that are infinite, i.e., the set of vectors x G K" with at least 
one component exceeding (M — lj2)A in magnitude. When x is such that the 
quantizer does not saturate, the quantization error satisfies the bound 

\\x- qA{x)\\< A^/n/2 (2) 

where || • || stands for the standard Euclidean norm. Observe that, according to 
the above terminology, x belongs to the saturation region of qA whenever at 
least one of the components of the vector qA{x) equals ±MA, even though this 
does not automatically imply that the inequality (2) fails to hold. 

Consider the linear control system 

X = Ax + Bu, x G M”, u G ]&"*■. (3) 



Suppose that it is stahilizable, which means that there exists a matrix K such 
that the eigenvalues of A + BK have negative real parts. Since the linear feed- 
back law u = Kx cannot be implemented, it seems logical to try the quantized 
feedback law u = KqA{x). As shown in [2], one can define a hybrid quantized 
feedback control policy based on this feedback law to render a; = 0 a global- 
ly asymptotically stable equilibrium of the continuous part of the closed-loop 
system. 

Theorem 1. [2] There exists a hybrid quantized feedback control policy that 
makes the system (3) globally asymptotically stable. 

Proof. The control law will take the form 



u{qA{x),z) 



0 if z = 1 

KqA{x) if z = — 1 



To define a desired control policy, we need to describe the evolution of A and z. 
Before proceeding to do that, we make some observations regarding the behavior 
of the system 



X = Ax + BKqA{x) 
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which can also be written as 

X = {A + BK)x — BK{x — qA{x)). (4) 

We will let Xmin{-) and Xmax{-) denote the smallest and the largest eigenval- 
ue of a symmetric matrix, respectively. Recall that by the standard Lyapunov 
stability theory there exist positive definite symmetric matrices Q and D such 
that {A + BK)"^Q + Q{A + BK) = —D. Whenever the inequality (2) holds, the 
derivative of x"^Qx along the solutions of (4) is given by 

^x^Qx = —x^Dx — 2x^QBK{x — qA{x)) 

< -Xmin{D)\\xf + 2\\x\\\\QBK\\Ay^/2 
= -||a:||(A™„(L>)||a;|| - \\QBK\\Ay^) 

The last expression is negative outside the ball {x : ||a:|| < OAyAi}, where 

e := \\QBK\\/Xmin{D). 

In other words, if qA does not saturate, we have 

||a:|| > OA^/n ^x^Qx < 0. (5) 

at 

In what follows we will use the simple facts that the radius of the ball in- 
scribed in an ellipsoid of the form {x : x"^Qx < 7 ^} equals 7 / \/ Xmax(Q) and the 
radius of the ball circumscribed about the same ellipsoid equals 'y/ \JXmin{Q)- 
Fix an arbitrary e > 0. Define the scaling factor f2 by the formula 




and take the saturation constant M of the quantizer qA to be large enough so 
that we have 17 < 1. 

We now describe the “zooming-out” stage of the control strategy {z = 1). 
Set the control to 0 and choose an arbitrary Z\(0) > 0. Then increase Z\ in a 
piecewise constant fashion, fast enough to dominate the rate of growth of ||e"^*||. 
For example, one can fix a positive number r and let Aft) = 7i(0) for t € [0, r), 
Aft) = for t G [r, 2r), Aft) = for t G [2r, 3r), and so 

on. Clearly, there will be a time t > 0 such that 



\x{t)\\<A{t) 






^min (Q) 
^max (Q) 



\\qAit)ixm\<m(^{M-^-)^ 



^min (Q) 
^max (Q) 




hence 
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by virtue of (2). We can thus pick a time to > 0 such that 
which implies that 

, 7 ) 

This inequality guarantees, in particular, that at time tg the quantizer does not 
saturate. It is important to note that the time instant tg was determined on the 
basis of the quantized measurements only. 

Next, we come to the “zooming-in” stage (z = —1). Starting at t = to, we let 
u = Kq^ix). We keep A equal to A{tg) until a later time to be specified below. 
It follows from (7) that x{tg) belongs to the ellipsoid 

^x : x'^ Qx < {A{tg))^ (^M - A™i„((5)|. (8) 

Since 17 < 1, it is not difficult to see that {M—l/2)^yXmin{Q) > XmaxiQ)- 

From this and (5) we conclude that x will not leave the ellipsoid (8) for as long 
as Z\ = A(tg), hence the quantizer will not saturate. Moreover, x will approach 
the smaller ellipsoid 

{x : x^Qx < {A{tg)Ye'^nXmax{Q)} 

(it might even happen that a; (to) already belongs to this ellipsoid). Thus we can 
pick a time ti > tg such that 

lk^„.,Wfi))ll < Mt„)(ev^+glA^ + A 

which implies that 



||a;(ti)|| < A{tg) 




^max{Q^ 

^min (Q) 




Therefore, x{ti) belongs to the ellipsoid 



{a; : x^Qx < (Z\(to))^ 





^max m- 



(9) 



Again, note that the time instant ti was selected using only the quantized mea- 
surements. 

The basic idea that allows us to achieve asymptotic stability is to decrease 
A by means of multiplying it by the scaling factor 17. Namely, we let A{t\) := 
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nA{to). Using (6), it is straightforward to verify that the ellipsoid (9) is identical 
to the one defined by (8) with f2A{to) in place of A{to). This means that we can 
continue the analysis for t >ti as before. (The fact that the scaling is performed 
at t = t\ is not crucial: since all the ellipsoids considered here are invariant 
regions for the closed-loop system, A could also be scaled at an arbitrary time 
t > ti.) Thus there exists a time t 2 > t\, which can be determined from the 
quantized measurements, such that x{t 2 ) belongs to the ellipsoid defined by (9) 
with A{ti) in place of A{to). When t = t 2 (or at an arbitrary time t > ^ 2 ) 
we set A := i7A{ti). Repeating this procedure, we obtain the desired control 
policy. Indeed, stability of the equilibrium a; = 0 in the sense of Lyapunov follows 
directly from the adjustment policy for A (note that the amount by which A 
needs to be increased initially is proportional to ||a:(0)||). Moreover, we have 
A{t) — > 0 as t ^ 00 , and by the above analysis the same is true for x{t). 
Figure 2 illustrates the proof. □ 





Fig. 2. Stabilization by hybrid quantized feedback 



In the preceding, A takes a countable set of values which is not assumed 
to be fixed in advance. In some situations A may be restricted to take values 
in some given set S. It is not hard to see that the proposed method, suitably 
modified, will still work in this case, provided that the set S satisfies the following 
properties: 

1. S contains a sequence An, A 21 , . . . that increases to 00 . 

2. Each Ail from this sequence belongs to a sequence An, Ai 2 , ... in S' that 

decreases to 0 and is such that we have 17 < for each j. 

If the set of possible values for A is finite rather than countable, we can only 
obtain practical stability and not global asymptotic stability. 

The above proof reflects just one among several different quantized feedback 
control strategies for linear systems presented in [2] . That paper treats the topics 
of achieving exponential convergence, quantized output feedback stabilization, 
quantized feedback stabilization under sampling, and stabilization using quantiz- 
ers with small saturation constants. Some of these questions are further pursued 
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in [11]. We have chosen the particular approach described here because it appears 
to be the most suitable one for generalization to nonlinear systems. 

3 Nonlinear Systems 

We now turn to nonlinear control systems of the form 

x = f{x,u), xGM", mGM™. (10) 

In this section, all vector fields and control laws are assumed to be sufficiently 
regular (e.g., smooth). Given a feedback law u = k{x), we consider the system 

X = f{x,k{x + e)) (11) 

where e is a measurement disturbance input, later to be associated with the 
quantization error. For the purposes of this paper, we will say that the sys- 
tem (11) is input-to- state stable (ISS) with respect to e if there exists a positive 
definite radially unbounded smooth function V : M” ^ M such that for some 
class /Coo functions^ ai, «2 and p, for all x ^ 0, and for all e we have 

ai(||a:||) < V"(a;) < a 2 (||a;||) (12) 

and 

Ikll > p(||e||) ^ VV{x)f{x,k{x + e)) < 0. (13) 

According to the results of [18], this is equivalent to the original definition of 
ISS given by Sontag in [15]. 

In this section we assume that the given system (10) has the property that 
there exists a feedback law u = k{x) which makes the system (11) input-to-state 
stable with respect to e. This is the property which we referred to as “external 
stabilizability” in the Introduction. We also assume that the functions ai, «2 
and p satisfy the following condition: 

(a^^ o ai)'(O) > 0 and p'(0) < oo (14) 

(there is no loss of generality in requiring that the above derivatives exist, if one 
allows the possibility that p'{0) might equal oo). We postpone a close examina- 
tion of these assumptions until Section 4. Everything that follows remains valid 
if one replaces a static control law u = k{x) by a dynamic control law. 

As before, let be a quantizer with sensitivity A and saturation constant 
M . The problem under consideration is to find a quantized state feedback law 
that makes the system (10) asymptotically stable. The idea that we propose is 
to use the above control law k, which results in the closed- loop system 

x = /(x, k{qA{x))) = f{x, k{x - {x - qA{x)))). (15) 

^ A function a : [0, oo) ^ [0, oo) is said to be of class /Coo if it is continuous, strictly 
increasing, and such that o;(0) = 0 and a(r) ^ oo as r ^ oo. 
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If QA does not saturate, then the inequality (2) holds, and we deduce from (13) 
that the derivative of V along solutions of the system (15) satisfies 

||a;|| > p(Zlv^/2) ^ 1/ < 0. (16) 

This fact will be used in the sequel in much the same way as the formula (5) has 
been used in the previous section. 

Fix a positive number e, and define the functions 

7 i(Z\) := o Qf 2 o p{A{y/nf2 + e)) + Ay/n 

and 

72 (Z\) := o ai{{M - l/2)A). 

Suppose for the moment that an upper bound on the initial state is known: 
||a^(0)|| ^ Eq, where Eg is a positive number. A desired hybrid quantized feedback 
control policy can then be described as follows. Choose an arbitrary Z\(0) > 0. 
In view of (14), an elementary argument shows that if M was taken to be large 
enough, then we have 



72(/i)>7i(^) VAg(0,A(0)] (17) 

and furthermore 

72(Z\(0)) > Ao. (18) 

It follows from (12) and (18) that a;(0) belongs to the region 

{x:V{x) <ai{{M -l/2)A{Q))}. (19) 

Using (16) and the inequality 72(A(0)) > 7i(Z\(0)), we conclude that x will 
not leave the region (19) for as long as A = Z\(0), and the quantizer will not 
saturate. Moreover, x must approach the smaller region 

{x :V{x)<a 2 0 p(Z\(0)v^/2)}. 

Thus we can pick a time > 0 such that 

lkzi(o)(a^(fi))ll < o 02 o p{A{0)y/n/2 + Z\(0)e) + Z\(0)^/n/2 

hence 

||a^(fi)|| < o 02 o p(A{0)y/n/2 + A(O)e) + A{0)^/n = 7i(Z\(0)). 

Therefore, x{ti) belongs to the region 

{x : U(x) < 02 °7i(^(0))}- (20) 

When t = ti, set A := 7^^o7i(Z\(0)). It is not hard to check that the region (20) 
is the same as the one given by (19) with Z\(0) replaced by this new value of 
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A. This means that we can continue the same analysis for t > ti. Repeating 
this procedure, we have Z\(t) — > 0 as t ^ oo because of (17), and asymptotic 
stability follows. The above argument is very similar to the one employed at the 
“zooming-in” stage in the proof of Theorem 1. 

Now suppose that there exists a value of M for which 

72(Zi)>7i(^) VZi>0. (21) 

In this case, an a priori bound on the initial state is not necessary, because 
we can apply a “zooming-out” procedure. Namely, suppose that the unforced 
system 



i = f{x,Q) (22) 

is forward complete, meaning that for every initial state a;(0) its solution, which 
we denote by ^(a;(0),t), is defined for all t > 0. Then we can set m = 0 and 
increase Z\ fast enough to dominate the rate of growth of ||a;(t)||. For example, 
take M large enough so that the function y(r) := 72 (r) -r-^n is of class /Coo - Fix 
a positive number r and let A{t) = x“^(max||a,(o)||<T- ||?(a^(0 ), t)||) for t G [0,r), 
A{t) = x“^(max||a;(o)||<2T ||^(a^(0)) 2r)||) for t G [r,2r), and so on. Then there 
will be a time t such that 



lk(/)|| < l 2 {A{t)) - A{t)y/n 

hence 

hA(t){x{t))\\ < j2{A{t)) - A{t)y^/2 
by (2). Thus we can pick a time to > 0 such that 

lkzi(to)(a;(/o))l! < i 2 {A{to)) - A{to)Vn/2 

hence 

||a:(to)|| < 72(^(/o))- 

This implies that a;(/o) belongs to the region (19), and from this point on the 
analysis can be continued exactly as before. We have thus proved the following 
theorem. 

Theorem 2. Suppose that the system (10) is input-to- state stabilizahle with 
respect to measurement disturbances, in the sense defined above. Suppose al- 
so that the condition (14) holds. Then for each number Eq > 0 there exists a 
hybrid quantized feedback control policy that makes the system (10) asymptoti- 
cally stable, with domain of attraction containing all initial states x(0) such that 
||a^(0)|| < Eo (semiglobal asymptotic stability). In addition, if the system (22) 
is forward complete and for some M the inequality (21) holds, then there exists 
a hybrid quantized feedback control policy that makes the system (10) globally 
asymptotically stable. 
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If the condition (14) does not hold, then the inequality (17) cannot be 
satisfied by choosing a sufficiently large M. However, for any given numbers 
Z\(0) > i5 > 0 there exists a positive integer M such that we have 

72(Z\) > 7i(/\) VZ\ e ((5, Z\(0)]. 

It is not difficult to see that in this case the above procedure gives semiglobal 
practical stability, in the sense that all initial states whose norm satisfies a known 
bound are driven to the region given by (20) with S in place of 2\(0). As explained 
in the next section, such a property can actually be achieved without imposing 
the input-to-state stabilizability assumption. 

We also point out that, in view of [19, Lemma 1.2], every asymptotically 
stabilizing feedback law is automatically input-to-state stabilizing with respect 
to the measurement error e locally, i.e., for sufficiently small values of x and e. 
This leads at once to local versions of our results. 

4 Discussion of Results 

The concept of input-to-state stability (ISS) captures the property that bounded 
inputs lead to bounded states, and inputs converging to zero produce states that 
converge to zero. This concept was introduced by Sontag in [15]. In the same 
paper he proved that if an affine system of the form 

X = f{x) + G{x)u 

is asymptotically stabilizable by a feedback law u = ko{x), then one can always 
find a feedback law u = k{x) that makes the system 

X = f{x) + G{x){k{x) + d) 

ISS with respect to an actuator disturbance d. However, there might not exist a 
feedback law that makes the system 

X = f{x) + G{x)k{x + e) 

ISS with respect to a measurement disturbance e, as was shown by way of coun- 
terexamples in [9] and later in [6]. Of course, for linear systems with linear 
feedback laws all three concepts (asymptotic stabilizability, input-to-state stabi- 
lizability with respect to actuator errors, and input-to-state stabilizability with 
respect to measurement errors) are equivalent. 

Thus the problem of finding control laws that achieve ISS with respect to 
measurement disturbances for the system (11) is a nontrivial one, even for affine 
systems. This problem has recently attracted considerable attention in the liter- 
ature (see [6,10,12]). In particular, it was shown in [10] that the class of systems 
that admit such control laws includes single-input plants in strict feedback form. 
As pointed out in [16], it also includes systems that admit globally Lipschitz 
control laws achieving ISS with respect to actuator disturbances, although this 
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condition is quite restrictive. In the paper [12] small-gain techniques are applied 
to handle certain classes of systems with unknown parameters and unmodeled 
dynamics. 

In Section 3 we showed how to achieve semiglobal or global asymptotic stabil- 
ity for nonlinear systems by means of quantized state feedback. The assumptions 
that were needed to prove the result included the existence of a feedback law 
achieving ISS with respect to measurement disturbances. We also imposed the 
condition (14) which characterizes the behavior near zero of the functions a\, «2 
and p that appear in the definition of ISS. To obtain global asymptotic stability, 
we required that there exist a saturation constant M for which the inequality 
(21) holds. This depends on the relative rate of growth of the above functions 
at infinity. We now give a simple example of a system for which all of these 
hypotheses are satisfied. 

Example. Consider the following system, which is a simplified version of the 
system treated in the example on page 811 in [12]: 

X = + XU, X, u G M. 

In [12] it is shown how to construct a feedback law k such that the closed- loop 
system 

X = x^ + xk{x + e) 

is ISS with respect to e. It follows from the analysis of [12] that the inequalities 
(12) and (13) hold with V{x) = x^/2, so one can take oi(r) = a 2 {r) = 
and for p one can take any linear function p(r) = cr with c > 1. We have 
(a^^ oQfi)'(O) = 1 and p'{Q) = c, so (14) is obviously valid. Moreover, (21) holds 
for every M > c{y/nj2 -I- e) -I- ^/n + 1/2. 

We conclude the paper with a discussion of how one can achieve semiglobal 
practical stability for a much more general class of nonlinear systems than the 
one considered in the previous section. The idea is based on the work reported 
by Sontag in [17] on designing stabilizing control laws that are robust with 
respect to small measurement errors. Suppose that the system (10) is globally 
asymptotically controllable to the origin, in the sense that every initial state x(0) 
can be driven to 0 by a bounded control. Every system that is stabilizable by 
continuous feedback certainly belongs to this category. The following statement 
is a direct consequence of [17, Theorem 1]: there is a class /Coo function T with 
the property that for any numbers Eq > e > 0 there exist positive numbers 
S{e,Eo), K{e,Eo) and T{e,Eo) and a feedback law u = k{x) such that for every 
measurement disturbance e with ||e(t)|| < k 6 Vt > 0 each solution of the system 
(11) with ||a:(0)|| < Eq satisfies 



\\x{t)\\<r{Eo) V/>0 


(23) 


and 




a;(t) < e Vt > T. 


(24) 


The feedback law k will in general be discontinuous. The solutions of (11) are 
then to be interpreted in the “closed-loop system sampling” sense as defined in 
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[4], with sampling period i5. Actually, the statement remains valid if the sampling 
period varies within the bounds S < S, provided that the inequalities 

S<S (25) 

and 

\\e{t)\\<kS (26) 

are satisfied. The proof of this result relies on the existence of a continuous 
(but in general not necessarily smooth) control Lyapunov function for a given 
asymptotically controllable system. The feedback law is defined in terms of all 
sampling schedules that are sufficiently fast, as expressed by the inequality (25). 
On the other hand, robustness with respect to small measurement errors is only 
assured if the sampling is not too fast, as expressed by the inequality (26). This 
modification is not necessary, however, if the system possesses a smooth control 
Lyapunov function (see [13]). The need for sampling disappears altogether if 
the stabilizing feedback law k is continuous, in which case the solutions of the 
closed-loop system can be interpreted in the classical sense. 

The above result suggests the following control strategy. Suppose that we 
know an upper bound on the initial state: ||a:(0)|| < Eq. Fix an arbitrary e G 
{0,Eo). Choose A small enough so that 

ZiVn/2 < k6 (27) 

and take M large enough to have 

r{Eo) <{M -l/2)A. (28) 

If we now take the above control law k and let u = k{qA{x)), then the quantizer 
will never saturate, and x{T) will be in a ball of radius e around the origin. We 
arrive at the following result. 

Proposition 3. Suppose that the system (10) is globally asymptotically control- 
lable to the origin. Then there is a class /Coo function E with the following prop- 
erty: for any numbers Eq > e > 0 there exists a quantized feedback control law 
such that, whenever ||a:(0)|| < Eq, the solution of the closed-loop system satisfies 
the estimates (23) and (24) for some T > 0 (semiglobal practical stability). 

One might also attempt to use a hybrid quantized feedback control policy, 
as in the previous sections, to drive x into a ball around 0 of a smaller radius 
e' < e. Namely, one can try to “zoom in”, i.e., replace A at time T by a smaller 
value for which the inequality (28) holds with e in place of Eq. The difficulty 
here is that both k and S in the inequality (27) depend on e and Eq, so it is 
in general not guaranteed that this inequality will still be satisfied with «:(e', e) 
and <5(e',e). Thus we see that the task of estimating the size of the smallest 
possible attractor requires a careful examination of the findings of [17], which is 
an interesting topic for further research. 
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Abstract. The paper deals with the diagnosis of continuous-variable or 
hybrid systems whose state can be measured only by means of a quan- 
tiser. Hence, the on-line information used in the diagnosis is given by 
the sequences of input and output events. The paper describes how the 
quantised system can be represented by a semi-Markov process and how 
the diagnostic problem can be solved by using this timed discrete-event 
representation. A specihc result is obtained if the model is does not 
include probabilistic information about the event occurrence. The diag- 
nostic method is illustrated by considering a numerical example which 
concerns a part of a batch process. The results show that the tempo- 
ral information included in the semi-Markov process is crucial for fault 
diagnosis of discrete-event systems. 



1 Introduction 

Diagnosis of quantised systems. This paper is concerned with the diagnosis 
of dynamical systems with discrete inputs and outputs. As shown in Fig. 1, 
the system under consideration is a continuous-variable continuous-time system 
that can be described by the state-space model 

X = f{x{t),u{t)J), x{Q) = Xo . ( 1 ) 

The system behaviour depends on the fault f G T where fo G T symbolises the 
faultless system. 

The system state x is accessible only through a quantiser, which generates 
an event whenever the state changes its qualitative value [a;] . Hence, the system 
output is a timed event sequence 

Et{0...th) = {Eo,To] Ei,Ti; E2,T2; ...;Eh,Th) (2) 

in which Ek denotes the name and Tk the occurrence time of the fc-th event, i/ 
is the number of events that the quantised system generates in the time interval 
[0,th]. 

The injector associates with a given input event sequence 

V{0...th) = (vo,Vi,V2,:;Vh) 



N. Lynch and B. Krogh (Eds.): HSCC 2000, LNCS 1790, pp. 258—271, 2000. 
(c) Springer- Verlag Berlin Heidelberg 2000 
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Fig. 1. Diagnosis of quantised systems 



the input u(t) of the continuous-variable system defined by 

u{t) = for tk <t < tfc+i ■ (3) 

The system consisting of the continuous-variable system, the quantiser and the 
injector is called the quantised system. 

The diagnostic problem is given as follows: 

Given: Model M of the quantised system 

Input and output event sequences Et{0...th) and V{0...th) 

Find: Fault / 

The development of a diagnostic method that solves this problem consists of 
two major steps: 

1. Modelling: A model M of the quantised system has to be found that is 
simple enough to be used in the diagnostic algorithm. Here, M is a semi- 
Markov process. In Sect. 5 it will be shown how this model can be set up for 
the quantised system. 

2. Diagnosis: A diagnostic algorithm has to be found for determining the fault 
/ for given input and output event sequences. In Sect. 6 such a diagnostic 
algorithm will be elaborated for the semi-Markov process M. 

After presenting the method for modelling hybrid systems by semi-Markov pro- 
cesses and for diagnosing quantised systems by using the semi-Markov process, 
specific results are obtained for timed nondeterministic representations that do 
not include probabilistic information about the quantised system. A comparison 
of the different results show that the temporal information about the quantised 
system is crucial for diagnosis. 



Relevant literature. Results along this line of research have been obtained 
in two fields. The modelling problem for quantised systems has been investi- 
gated, for example, in [2], [3], [10] or [13]. As an extension to these results, the 
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model proposed in [4] is used here. It describes the timed event sequences and, 
thus, includes more information about the quantised system. On the other hand, 
diagnosing quantised systems by means of a discrete-event representation has 
been investigated in [1], [7], [8], [11] or [12]. This paper extends these methods 
to timed discrete-event representations. The proof of the diagnostic algorithm 
can be found in [5]. 

2 Example: Diagnosis of a Batch Process 

The class of diagnostic problems considered in this paper is illustrated by the 
batch process depicted in Fig. 2. The tank system is a hybrid systems because 
the dynamic properties are switched depending on the current liquid levels. The 
dashed lines mark liquid levels, which are measured by sensors that indicate only 
if the level is higher or lower than its position. These sensors act as quantisers. 



The following operation from a batch process is considered. At t = 0 the 
liquid level in Tank 1 (left) is ’’high” (i.e. higher than the dashed line) and the 
level in Tank 2 is ’’low” . The aim is to bring the level in the right tank above the 
upper dashed line. To do this, the Valves Vi, V2 and V4 are opened and Valve 
V3 closed. 

The only on-line information is obtained from the qualitative sensors posi- 
tioned at the dashed lines in Fig. 2. Consequently, the behaviour of the system is 
considered in the partitioned state space depicted in Fig. 3. If the trajectory of 
the system crosses one of the partition borders, an event is generated. Figure 3 
shows as two examples the events 634 and 643. 

The fault set IF = {/o, /i, /2, /s, /4} is considered where /i, /2 and /4 denote 
the situation that the Valve Vi, V2 or V4 is not opened, respectively, and fs 
describes that Valve V3 is not closed. The diagnostic problem is to find the fault 
as quickly as possible after the control input, which opens the valves Vi, V2 and 
V4, has been applied. 




hi 




Tank 1 



Tank 2 



Fig. 2. Example of a batch process 



Diagnosis of Quantised Systems 261 




hi 



Fig. 3. Partition of the state space of the two tanks (xi = level of 
Tank 1; X 2 = level of Tank 2) 



3 Quantised Continuous— Variable Systems 

The quantised system consists of the continuous-variable system (1), the quan- 
tiser and the injector. It is assumed that for any initial state Xq and input u{t) 
(1) has a unique solution, which will be considered for the time interval 
and denoted by 



Quantisation of the state space. The quantiser maps the state space IR” 
onto a finite set Afx = {0, 1, 2, N} of qualitative values and, thus, introduces 
a partition of IR" into a finite number of disjoint sets Qx{i), where Qx{i) denotes 
the set of states x G IR" with the same qualitative value i. The mapping invoked 
by the quantiser is symbolised by the symbol [.]: 

[x] = i X e Qx(i) ■ (4) 

The sets Qx{i) {i = 1, are assumed to be bounded while Qa;(0) is the un- 
bounded ’’remaining” subset of IR": Qa;(0) = IR"\ • For the bounded 

sets, SQx{i) denotes the hull of Qx{i)- Figure 3 illustrates the state quantisation 
for the example batch process. 

The quantised system is said to generate the event ey at time tk if 

[x{tk + St)] = i and [x{tk — i5t)] = j (5) 

hold for some i,j G Afx,i yf j for arbitrarily small St > 0. In this way, the 
timed event sequence (2) is obtained. The relation between a continuous state 
trajectory and the event sequence Et is given by the quantiser, which is 

symbolised by 

Et{0...th) = QuanQ (a;[o,t^]) ■ (6) 



Injector. The injector associates with the discrete input value [n(f)] = v {v G 
Nu) a quantitative value u" gU <G IR"*. It is assumed that the input u changes 
its value simultaneously with the qualitative state [a;]. In this way, a given input 
event sequence V is transformed into the input function (3). 
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4 The Modelling Problem 

For the diagnosis, a model has to be used which generates for every given initial 
event eg and input event sequence V the event sequence Et{0...th) for all faults 
f € T. Such a model is available if (1) is combined with the quantiser and the 
injector. However, this model includes continuous-variable and discrete-event 
parts. For diagnosis, a more compact model has to be found. 



Nondeterminism of the discrete event behaviour. An inherent problem 
of this modelling task results from the from the fact that these event sequences 
are not unique [6]. The nondeterminism of the discrete-event behaviour means 
that the quantised system may generate one of a set of different event sequences 
Et and it is impossible to select the true sequence in advance. The reason for 
this is given by the fact that the initial state Xg of the system (1) is unknown. 
After the first event eg has been observed at time tg, the state of the system is 
known to lie in a subset 6Q{eg) of the corresponding partition border . If, for 
notational convenience, tg is assumed to be zero, the occurrence of the event eg 
restricts Xq to be in the set SQ(eg) at that time instant. Depending on Xg the 
system may produce one event sequence of the set 

St(eo,V(0...th),f) = = Quant | Eqns. (1), (3) hold for some 

Moreover, the temporal distance of the events may vary considerably. 



Stochastic properties of the quantised system. A compact representation 
of the nondeterministic behaviour of the quantised system can be obtained by a 
statistical evaluation. It is assumed that the initial state Xg of the continuous- 
variable system (1) is uniformly distributed over the set 6Q{eg). Then the event 
sequence Et is a random sequence with Et G St(eg, V, /). The probability that 
the event e has occurred before or at time t is denoted by 

Ve(e,t,f) = J2Proh(Ek=e,n<tlF=f) . (8) 

k 

Figure 4 shows the statistical properties of the quantised tank system. The 
strips depict the probability I4(e,t, /i) in grey scale. The strips are shown only 
for the time interval in which > 0 holds, because the event e may occur 
exactly in this time interval. The darker the strip is the more probable is the 
occurrence of the event until the corresponding time instant. The numbers on 
the right margin show the final probability values. 



Modelling aim. The model, which will be used for diagnosis, should describe 
the relation between the initial event eg, the qualitative input sequence V (0...th) 
and the event sequence Et{0...th) for all faults f G E. Since the behaviour is 
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Fig. 4. Graphical representation of the statistical properties of the 
tank system for fault /i and initial event 642 



nondeterministic, a nondeterministic model has to be used which generates a set 
Aft(eo, V, /) of event sequences. The modelling aim is to find a representation 
such that the relation 



Mt{eo,V{0...th),f) D St{eo,V{0..th)J) 



(9) 



holds for all cq, V, f and th- That is, the model is complete in the sense that it 
generates all event sequences that the quantised system may generate. 



5 Representation of Quantised Systems 
by Semi— Markov Process 

5.1 Brief Introduction to Semi Markov Processes 

In a semi-Markov process Mr{Z, V, /r, zq), Z is the set of states, V the set of 
input values (input alphabet), fq- the probability density function and zq the 
initial state. The semi-Markov process changes its state Z instantaneously at 
the time instances Tk {k = so that the process can be described by 

the sequence 



Zt{0...th) = {Zo,To] Zi,Ti; Z2,T2] ...;Zh,Th) , (10) 

which means that the process assumes state Zk in the time interval [Tk,Tk+i). 

As the state of the semi-Markov process cannot be unambiguously predicted, 
the process is described by the state probability pt{z, t) = Prob {Z{t) = z). It is 
assumed that the input V (t) of the semi-Markov process changes simultaneously 
with the state such that V (t) = Vk for Tk < t < Tk+i- 

In order to determine pt{z,t) for t > 0, the transition relation between any 
pair of states z,z G Z,z ^ z has to be described. This is done in terms of the 
sojourn time t = Tk — Tfe-i by the probability distribution 

F:^s{t,v) =Froh{Zi = z,Ti < t\Zo = z,To = 0,Vo = v) for z ^ z. (11) 

of the semi-Markov process, which is assumed to be homogeneous. 

The semi-Markov process can generate any trajectory (10) for which any 
pair (zfe_|_i, Zfe, tk) of successive states can occur with non-vanishing proba- 
bility. Hence, the set of trajectories with length H that the semi-Markov process 
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generates when starting in the initial state zq under the influence of the input 
sequence V{Q...th) is given by 

Mt{zo,V{Q...th)) = {( 2 : 0 , 0 ; zi,ti] ...;zH,th) \ /zfc-|_lZfc (tfc+l > 0 

holds for fc = 0, 1, i? — 1} (12) 

with 

fzz{T,v) = —F:,s{t,v) . 

5.2 Representation of the Quantised System 
by a Semi Markov Process 

This section extends the results of [4] for describing quantised systems by a 
semi-Markov process to systems with inputs and faults. Lemma 1 states how 
the probability density /r of the semi-Markov process has to be chosen for a 
given quantised system in order to satisfy the modelling aim (9). 

The semi-Markov process Mr{£, V, F, fr, cq) is now used with Z = £, where 
£ is the set of all events that the quantised system may generate. Thus, all 
relations of the preceeding section can be written with e replacing 2 . The fault 
/ considered occurs as new argument in all functions that have been introduced 
for the semi-Markov process, particularly in the probability density 

fr ■■ £ X £ x\R+ xV X F — > [0, 1] 

which will be referred to by the abbreviation 

/efc+ie,(r,t>,/) = /r(efc+i,efe,T,u,/) . 

In order to satisfy the modelling aim (9) the probability density function of 
the semi-Markov model has to be chosen according to the relation 

(13) 



On the right-hand side, a pair (e, e) of succeeding events is considered and the 
probability of its occurrence determined by means of (1), (3) for given v and /. 

Lemma 1 The semi-Markov process Mq-{£, V, T, fr, eo) satisfies the modelling 
aim (9) if the probability density fr satisfies (13). 

This lemma follows from Theorem 1 in [4] if the model is considered for fixed 
z; G V and / G IF. The set Mt{eo,V{0...th), f) generated by the semi-Markov 
process is given by 

Mt{eo,V{0...th),f) = |(eo,0; ei,ti; ...;ei/,t//) | /efc+iCfc 

holds for /c = 0, 1 , H — 1} . 






Diagnosis of Quantised Systems 265 



6 Diagnosis of the Quantised System 

6.1 The Principle of Consistency Based Diagnosis 

The diagnostic problem can be posed as the following question: 

Can the quantised system generate the event sequence Et{0...th) if it 
has obtained the input sequence V{0...th), i.e. does the relation 

Et{0...th)€St{eo,V{0...th),f) (14) 

hold for some / € E? 

Note that cq on the right-hand side of (14) is the first element of Et on the 
left-hand side. The diagnostic result is denoted by p{f,th) as follows: 

p(f, th) > 0 if Et(0...th) G 5t(eo, V{0...th)J) . . 

p{f,th) =0 else . '' 

p{fEh) > 0 says that the observed behaviour over the time horizon is 

consistent with the quantised system and p{f,th) = 0 means that the fault / 
cannot have occurred. 



6.2 Diagnosis of Semi Markov Processes 

The diagnostic problem is first solved for the model Mr{£,V,E,fT,eo). The 
result is denoted by 

PM{f,th) = Pmh{f \Et{0...th),V{0...th)) , (16) 

so that the relation 

PM{f,th)>0li Et{0...th) & Mt{eo,V{0...th),f) 

PM{f,th) =0 else '' 



holds. 
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Fig. 5. State sequence of the semi-Markov process 



The solution will be described with the symbols defined in Figure 5. The 
input and state sequences are considered for the closed time interval It is 

assumed that H is the number of events that occurred in the open time interval 
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The event €h was generated at time tn- vh is the input to the system 
for t > tn- 

For the solution of the diagnostic problem two cases have to be distinguished: 

Case (a): At time th the {H + l)-st event ch+i occurs, i.e. Ih+i = th 
Case (b): There is no event occurring at time th, i.e. tn+i > th ■ 

Since the fault may occur at any time t < 0 it may or may not influence the 
initial event cq, which is assumed to occur at time to = 0. Hence, 

Pm(/, 0) = — for all / G (18) 

np 

is used where np denotes the number of faults considered. 

The diagnostic result is obtained for time th by first determining an auxiliary 
function pa- 



Case (a): Pa{f,th) fen+ien {th - tn, Vh, /) PmU, tn) 
Case (b): Pa{f,th) = FeH{th — tH,VH,f) PM{f,tH) 

with 



Fee{T,V,f) 




fee{T,V,f)dT 



Fe{T,V,f) = l- ^ Fge{T,V,f). 



Second, the diagnostic result is obtained from 



Pm(/, th) 



Pa{f,th) 
'^fepPaif, th) 



provided that 

^Pa{f,th) > 0 

/e^ 

holds. 



(19) 



( 20 ) 



(21) 



( 22 ) 



Theorem 1 PM{f,th) obtained by (21) describes the probability (16) that the 
output sequence Et{0...th) has been generated for the input sequence V{0...th) 
by the semi-Markov process with fault f. 

If (22) is violated for some time th the event sequence Et is inconsistent with 
the semi-Markov process for all f G F. The proof is given in [5] . 



6.3 Diagnosis of Quantised Systems 

The algorithm described in Sect. 6 is now used to solve the diagnostic problem 
for the quantised system. This can be done due to the following theorem [5]: 
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Theorem 2 Assume that the diagnostic algorithm (18) - (21) has been applied 
to the semi-Markov process Mq- for a given input sequence V{0...th) and event 
sequence Et{0...th). The relation 

PM(f,th) = 0 implies p{f,th)=0 (23) 

holds if and only if the model is complete and, thus, the requirement (9) is sat- 
isfied. 

Theorem 2 shows that the semi-Markov process, which is obtained from the 
abstraction operation described in (13) can be used for diagnosing the quantised 
system. Equation (23) yields the following corollary. 

Corollary 1 The diagnostic algorithm (18) - (21) which is applied to the semi- 
Markov process that satisfies the relation (13) yields the following results: 

— Fault detection: If PM{fo,th) = 0 holds (where fo symbolises the faultless 
system), then some fault has occurred in the quantised system. 

— Fault identification: If PM(fAh) = 0 holds, the quantised system has not 
been effected by the fault f. 

If pM{f,th) 7 ^ 0 holds, the fault / may have occurred. Fault identification by 
consistency-based diagnosis means to exclude those faults that, according to the 
information available, is known not to have occurred. 

6.4 Diagnosis by Means of a Nondeterministic Representation 
of the Quantised System 

In this section, the diagnosis will be considered under the assumption that the 
probabilistic information included in the probability density function fq- of the 
semi-Markov process is not available. Then the timed description provides only 
time intervals 

/) = [tminee(^j /)i tmaxee(^j /)] 

with the upper and lower bounds tmaxee and trainee of the time that passes 
after the event e before the successor event e is generated by the quantised sys- 
tem. This result is interesting for three reasons. First, if the model (1) of the 
continuous-variable system is not available, experiments made with the quan- 
tised system can bring about the information required to determine the time 
interval 7)^, whereas these experiments may be insufficient to provide the prob- 
abilistic information included in fq-. Second, if the diagnostic results obtained 
for the semi-Markov process and this nondeterministic representation are com- 
pared it becomes obvious whether the probabilistic or the temporal information 
included in the semi-Markov process is of more importance for the efficiency of 
the diagnosis. Third, in discrete-event systems theory models are used which 
describe the temporal distance of events by time intervals, for example, time- 
labelled Petri nets. In [13] a method is described for obtaining such a model for 
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quantised systems. Such models can be used in the following diagnostic algo- 
rithm. 

If the semi-Markov process is given, the borders of the time interval T^e can 
be determined as follows: 



fminee(v, /) = min /ee(r, U, /) yf 0 

r 

fmaxee (^5 /*) — max f ^ V ^ f 'j yf 0 . 
r 

However, it can be determined also experimentally by measuring the time that 
passes between the events e and e if the quantised system with input v and fault 
/ has different initial states Xg- 

The diagnostic result PM{f,th) is no longer the probability of the fault oc- 
currence, but it only shows whether the fault / can be diagnosed until time th 
{PM{f,th) = 1) or not {pM{f,th) = 0). Therefore, the following modifications 
have to be made for the diagnostic algorithm. The initial values are 

Pm(/,0) = 1 for all / G (24) 

because no fault can be excluded without any on-line information about the 
quantised system. The auxiliary function pa says whether the quantised system 
subject to fault / and input vh can generate the event en+i after the sojourn 
time th — tn- 



1 if PmU, tn) = 1 and th-tn & %H+ieH /) 

0 else 

1 if PmU, tn) = 1 and th-tn € Te„ {vh, /) 

0 else 

(25) 

Tejj describes which time may pass until the quantised system generates the 
successor event of e_y: 



Case (a): Pa{f,th) = 
Case (b): Pa{f,th) = 



'^en (v, f) = {t\3t > t,e : t G Tees } ■ 
The diagnostic result is obtained from 

PM{f,th) =Pa{f,th) 



provided that 

^Paif.th) > 0 

/e^ 

holds. 



(26) 

(27) 



7 Example: Diagnostic Results for the Batch Process 

The diagnostic algorithm is now applied to the batch process. Figure 6 compares 
the event sequences that the tank system generates for the different faults with 
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0 21 42 63 84 105 126 147 168 189 210 

Time in seconds 



Fig. 6. Comparison of the discrete-event behaviour for different 

faults 




Fig. 7. Discrete-event behaviour of the process subject to fault /i 
and diagnostic result 



initial event 642 - Note that the event sequence ( 642 , 634 , 653 , 655 ) may be gener- 
ated by the faultless system as well as by the system with the faults fi, /2 or f^. 
Therefore, fault diagnosis is possible only if the temporal distance of the events 
are taken into account. 

In the upper part of Fig. 7 the discrete-event behaviour of the tank system 
is presented for fault fi. The dashes show at which time the events 612 , 631 , 653 , 
and 655 occur. These time instants are marked in the lower part of the figure by 
dotted lines. The lower part shows the diagnostic result, where the probability 
PM{f,th) is depicted in grey scale. Obviously, the fault /i is uniquely detected 
after about 30sec. That is, PM{fi,th) = 1 holds for th > 30, which is also 
indicated at the right margin of the figure. Note that the diagnosis is finished 
before the second event occurs. 

The figures show how the fault probabilities change over time. In practical 
applications, a threshold will be used and a fault is announced only if its proba- 






270 



J. Lunze 



bility exceeds this threshold. This, however, includes some heuristics concerning 
the threshold level, which is not the subject of this paper. 




Zeit in s 



Fig. 8. Comparison of the diagnostic result obtained by means of the 
semi-Markov model (top) and the untimed stochastic automaton 

(bottom) 



If the nondeterministic model without probabilistic information is used in the 
diagnosis, the result is the same with the only difference that all stripes in Fig. 7 
are black rather than grey. Consequently, the results of fault identification are 
the same. The additional probabilistic information included in the semi-Markov 
process makes it possible to distinguish all fault, which cannot be excluded, 
concerning the degree of certainty with which they exist in the quantised system. 
This degree of certainty is described by the different grey levels in the figure. 

Figure 8 shows a comparison with the diagnostic result obtained by using an 
untimed description of the quantised system. Obviously, the result with the timed 
model is much better. This demonstrates the fact that the temporal information 
included in the semi-Markov model is the key information for fault identification. 



Conclusions 

The paper has presented a method for diagnosing quantised continuous-variable 
systems. The method is based on a timed discrete-event representation of the 
quantised system by means of a semi-Markov process. It has been shown how 
the probability density function of the semi-Markov process can be obtained 
from the quantised system and how this model can be used for diagnosis. The 
diagnostic algorithm is very simple. It includes only some multiplications to be 
carried out in each recursion step. The simplicity of the algorithm is based on the 
simplification of the model, which has been introduced by the timed abstraction. 
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Abstract. The paper deals with the qualitative analysis of the so-called 
switched flow networks. Such networks are used to model various commu- 
nication, computer, and flexible manufacturing systems. We prove that 
for any deterministic network from a specific class, there exists a finite 
number of limit cycles attracting all the trajectories. Furthermore, we 
determine this number. 



1 Introduction 

The paper considers hybrid dynamical systems that are called switched flow 
networks. Special classes of such networks were introduced in [7] to model flexible 
manufacturing/assembly/disassembly systems. These networks are also useful to 
model various computer and communication systems, especially those with time- 
sharing schemes. Other examples concern batch processes, chemical kinetics, and 
biotechnological processes. 

As is known, even very simple flow networks of the second order can exhibit a 
chaotic, irregular, unpredictable behavior [1,2]. Such a behavior is unacceptable 
for most of real systems. A typical synthesis problem (see [2,10,11,3]) is to And 
a feedback switching policy that ensures a regular, predictable behavior of the 
network. Dealing with this problem involves qualitative analysis of the dynamics 
of the close-loop system. Up to now samples of such investigations [1,2,10,3] were 
mainly confined to specific two-dimensional systems. The main idea underlying 
the theoretical analysis was reduction to iterated maps of an interval into itself. 

The network studied in this paper consists of buffers (nodes) connected with 
links (edges). We refer to the content of buffers as ’’work”; it will be convenient 
to think of work as a fluid, and a buffer as a tank. (In applications, ’’work” may 
represent a continuous approximation of a discrete flow of jobs in a computer 
system or parts in a manufacturing system, etc.) Work arrives from outside the 
system at fixed rates at certain buffers. The network is processed by a single 

* This work was supported by the Australian Research Council 
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server, which is able to deal with only one buffer at any moment. The server 
removes work from a selected buffer and delivers it at fixed rates along the edges 
departing from this buffer. The location of the server is a discrete control variable 
determined by a feedback policy. 

We consider quite general networks of arbitrary dimension. More precisely, 
we assume that the network may have an arbitrary number of nodes and any 
node may have an arbitrary number of edges both departing from and arriving 
at it. Nevertheless, an edge coming from inside the system and one coming from 
outside it cannot arrive at a common node. Furthermore, we suppose that the 
network contains neither cycles nor impasses, i.e., for any node, there exists 
an edge both arriving at and departing from it. We consider a deterministic 
network; more precisely, the rates at which work is transferred along the edges 
are assumed to be constant and fixed. This model generalizes in particular those 
from [1,3], where the case of three buffers with no edges between them and 
certain specific control policies was studied. 

We show that, depending on the system parameters, either 1) the total 
amount of work in the buffers converges to infinity in course of time for any 
switching policy, or 2) no policy can keep the system working for a long time, 
so far as infinitely many buffer changes accumulate at the vicinity of a finite 
time instant, or 3) a scaled total amount of work in the buffers remains con- 
stant whatever control policy be adopted. (Underscore that the statement 2) 
concerns the fluid model of the network. At the same time, accumulation of 
buffer changes signals that the conditions under which the continuous (fluid) 
approximation can be employed to model the real (discrete) network are vio- 
lated. So the conclusion in question certainly cannot be directly extended on 
the real-life discrete prototype of the model at hand.) The further consideration 
is focused on the case 3). We study a natural switching strategy that extends 
the so-called Clear-the-Largest-Buffer-Level [7] one. Our main result is that the 
close-loop system exhibits a periodic behavior almost always, i.e., whenever the 
tuple p of its parameters lies outside a certain set E of the zero measure. More 
precisely, there exists a finite number of limit cycles each being locally asymp- 
totically stable, and any trajectory converges to some of them. Furthermore we 
count these cycles and discuss phenomena that occur if p G E. 

To obtain criteria for existence of self-excited oscillations or limit cycles is 
an old and challenging problem of the classic qualitative theory of differential 
equations whose origins may be traced back to the work of Poincare and Lya- 
punov (see e.g. [6]). Few constructive results are known for nonlinear systems 
of order higher than 2. It is even harder to study stability of limit cycles. Our 
result shows that constructive criteria for existence and global stability of limit 
cycles can be proved for quite general switched flow networks. This appears to 
be surprising and gives us a hope that it is possible to develop a qualitative 
theory of some classes of hybrid dynamical systems, which will be even more 
constructive than the classic qualitative theory of differential equations. 

The ideas underlying the proofs of the results presented are related to the 
general theory developed in [4, 8, 5, 9]. 
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2 Single Server Flow Networks 

Consider an oriented graph with the set of the nodes 

G-.= {gi,...,gL,9L+i = oo}. 

The edge departing from gi and arriving at gj is denoted by {gi,gj). (There is 
no more than one such edge.) The special node oo is interpreted as the exterior 
of the system. Correspondingly, any edge of the form {oo,gi) and {gi, oo) (where 
i = 1, . . . , L) is regarded as coming from outside and going outside the system, 
respectively. 

Assumption 1 The graph satisfies the following properties: 

— If {oo,gi) (i = is an edge, there is no other edge arriving at the 

same node gi. 

— The graph contains no cycles. (In particular, {g,g) is not an edge for any 
g G G.) 

— For any node gj (j = 1,...,L), there is an edge arriving at gj, as well as 
that departing from gj . 

Associated with each node 



g & G := {gi,...,gL} 

is a buffer (or tank). Its content is called ’’work” and interpreted as fluid. The 
work arrives to the system continuously along the edges of the form {oo, g) at a 
constant rate pg > 0. There also is a server (or machine), which serves buffers. 
At any time, the server is able to deal with only one buffer. While so doing with 
a specific buffer g, the server removes work at a constant rate Sg > 0 and delivers 
it along the edges departing from g. The distribution of the work flow among 
the edges is in a given proportion. In other words, the server sends work along 
the edge {g,g') at a constant rate Sgp{g,g'), where p{g,g') > 0 and 

P(5.5') = l '^g&G. (1) 

g'&G(g) 




Fig. 1. A flow network. 
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Here 

G{g) ■■= js' G G : (g, g') is an edge | . 

The location of the server is a control variable, which is chosen in accordance 
with a prescribed feedback control policy. We assume that the server switches 
between buffers instantaneously. 

Depending on the system’s parameters, certain dynamical properties can or 
cannot be ensured by choice of a switching policy. To specify this statement, we 
introduce some notations. Put 

Gr{g) := G( 5 ) \ {oo}, := {5 G G : Gr{g) = 0} . 

Introduce also the sets S- 2 , S- 3 , ... by setting iteratively for i = —1, —2, . . ., 

:= {g G G : Gr(g) C 6 i := S'i U Sj+i U . . . U S'-! and g€&i}. (2) 

As can be easily shown, the sets Si are pair-wise disjoint and there exists an 
integer N such that S-i = 0 for all i > N, S-i yf 0, . . . , S-n yf 0. Furthermore 

G = S'-! U S'_2 U . . . U S-N- 

Next, we define a number <5^ > 0 for any node g G G. We first put Sg := s~^ for 
all g G S'-!. Suppose that the number Sg has been defined for all g G & i. Then 
we put 

^9'P(9,g') ^gGS-i-i. (3) 

g'eGr(g) 

(In the sum on the right, the multiplier 6 g> is already defined in view of (2) and 
the induction hypothesis.) Finally, we introduce the set of the nodes at which 
work arrives from outside the system 

91 := {g G G : ( 00 , g) is an edge } . (4) 

Denote by Xg the content of the buffer g. 

Lemma 1. Assume that the network contains at least two bujfers. Suppose also 
that 

^gPg >1- (5) 

g&m 

Then the total amount of work in the system 

w{t) ■= 

geG 

converges to 00 as t ^ 00 . If on the contrary 

Y^ ^gPg ^ 

gem 



( 6 ) 
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infinitely many buffer changes accumulate at the vicinity of a finite time t* > 0. 
These assertions are true irrespective of what control policy he adopted.* 

Thus no control policy can make the system even dissipative (in the sense that 
limsupj^oo w{t) < oo) if (5) holds. If on the contrary (6) is true, no control pol- 
icy can keep the system working for a long time. (We underscore once more that 
Lemma 1 concerns the fluid model of the network. At the same time, accumula- 
tion of buffer changes means that while dealing with a specific buffer, the server 
processes an amount of work that tends to zero. This apparently contradicts the 
conditions under which the continuous fluid-like ’’work” can be considered as 
a proper model of a discrete and quantified flow of jobs in a computer system 
or parts in a manufacturing one etc. So in the case (6), the conclusions of the 
lemma cannot be directly applied to the real-life prototype of the model at hand 
if this prototype is discrete in its nature.) Further we consider the case where 

SgPg = 1- (7) 

g&m 

From now on this relation is assumed to be valid. It is easy to see that then the 
scaled total amount of work in the system 

g&G 

remains constant in the course of time. 



3 A Switching Control Policy 

In this paper, the network is regarded as composed of receiving and processing 
parts. The first one is constituted by the nodes from the set (4), the second one 
fp consists of the rest of the nodes (except for the ’’exterior” one oo) := G\ 91 . 
We assume that the server processes these parts separately on the base of the 
Clear-the-Largest-Buffer-Level policy [7]. Thus its work splits into consecutive 
sessions of serving either the receiving or the processing parts, respectively. More 
precisely, we consider the following switching policy. 

SPl The server starts with the receiving part of the network. 

SP2 This part is served on the basis of the Clear-the-Largest-Buffer-Level strat- 
egy [7] . This means that the server switches when the current buffer is emp- 
tied and to a buffer g G with the largest (over 91 ) scaled content (g := 
CgXg. (Here Cg > Q is a given scaled coefficient.) Likewise, the server starts 

* We assume, however, that the server is working constantly, i.e., there are no periods 
when it is standing idle. Note also that t* < t, where the time t = t[w(0)] is inde- 
pendent of the switching policy. Any policy that makes the system working for the 
longest possible time t clears up the network in the sense that w(t) —> 0 as t —> t — 0. 
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to process the part in question with a buffer g G having the largest value 

SP3 The server ends to deal with the receiving part when it has changed buffers 
(fc — 1) times (within the current session) and has emptied the kth buffer 
g G ■ (Here k is the number of the buffers in the receiving part ‘TK .) 

SP4 After this, the server enters the processing part of the network and serves 
it on the basis of a similar strategy. More precisely, the server switches when 
the current buffer g G ^ = G\V\ is emptied and to a buffer g G with the 
largest (over ) scaled content (g := CgXg. Likewise, it starts with a buffer 
g G ^ having the largest value of (g. 

SP5 The server deals with the proeessing part of the network until it becomes 
empty Xg = 0 \/g G .* * * Then it returns to the receiving part of the 
network. 

In some cases, the server can be switched to an empty buffer in accordance 
with the above policy. (Such a case may occur only at the first service session 
and for special initial data.) Then the next server switching is implemented 
immediately. Thus the server can make several instantaneous buffer changes 
until it reaches a nonempty one. 

The state of the system is described by a pair {x,q) consisting of the ’’con- 
tinuous” X and ’’discrete” q components. Here x = {xg}g^c and 

qGQ-= g^G,i^i,2,...- 

Being in the state {g, i), where either g G ‘TK or g G ^ , means that the server is 
dealing with the buffer g and this buffer is ith in the current session of serving 
the receiving or the processing parts of the network, respectively. 

The evolution of the system is described by the following logic-differential 
equations: 



if q={g,i) then 



Xg' = pg! whenever g' G iR and g' g 
■ _ { —Sg if gG 

< ~ \pg - Sg if g G fR 

Xg' = Sgp{g,g') ifg' G Gr{g) 

Xg! = 0 otherwise 



(9) 



if q{t) = {g,i),g gG, i = 1,2,. . . and Xg{t) = f) then 



** If the largest scaled content is attained at several buffers, there is a variety of can- 
didate buffers to be switched to. Though there is no reason to prefer any of them, 
one can do so by specifying the control policy. We, however, consider all the pos- 
sible decisions. Therefore in the event in question, several continuations of a given 
trajectory are taken into account. 

* * * Note that a given buffer g G can be visited several times during one session. 
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where (/' G 

( 5 ',i+l)ifge 9^ andz</c 1 is such that 

(s', 1) if s G and ^ Xg„ < 0 \ 

g"e<p J 

Vs" G 9^ 
where s' G ^ 

(s', t + 1) if s G and ^ a;,. >0 ) 

g"e<p > 

(s', 1) if s G 9^ and i = k J — Cg"(^) 

Vs" G *P 

s(0) = (s, 1) where s G 91 is such that Cg(0) > Cs'(O) '^d' V 9^ . 

Except for the events specified, the discrete state q{t) keeps its value in course 
of time. 

Strictly speaking, the second formula holds only if the buffer corresponding 
to q{t + 0) is not empty. Otherwise, several buffer changes are performed instan- 
taneously at the time t and the formula for q{t + 0) must be modified. We omit 
the details so far as, on the one hand, they are apparent and, on the other hand, 
the event in question is not typical: it may occur only at the first service session 
and for initial data from a set of the zero measure. 

Any pair of functions [a;(-), g(-)] with x{-) absolute continuous and g(-) piece- 
wise constant and left-continuous that satisfy the above equations is called a 
trajectory. A given initial data may give rise to several trajectories since the 
buffer g with the largest scaled content (jg is not determined uniquely in certain 
cases. A simple analysis shows that any trajectory can be extended on an infi- 
nite time interval.^ (From now on, we consider trajectories defined on such an 
interval.) Furthermore the times of discrete state transitions do not accumulate 
and, being put in ascending order, form an infinite sequence {tn\jjj=i such that 
^ oo as n ^ oo. Supplemented by the term tg := 0, this sequence is called 
the switching time sequence of the trajectory. 

We assume that (7) holds and consider trajectories with cr(0) = 1, where the 
quantity cr is given by (8). The system is studied in the invariant domain 

K := {{x,q) ■. q & Q, Xg > 0 Vg, cr = 1} . (10) 

4 Asymptotic Behavior of the System 

For X = {a;g}ggG {xg G M), we put ||a;|| := l^sl- symbol mes stands 

for the Lebesgue measure. We start the section with several definitions from [5]. 




^ We recall that the case (7) is considered. 
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Definition 1. Let [x(t),q{t)] be a periodic trajectory, T > 0 be its minimal 
period, and let {tk} be its switching time sequence. An integer s is said to be the 
order of this periodic trajectory if 

ts <T < ts+l- 

It is easy to see that tis+j = IT + tj for ^ = 0, 1, . . . and j = 1, 2, — 

Definition 2. Let tp = 9p(')] ® periodic trajectory, {tk} be its switch- 

ing time sequence, and let s be its order. Furthermore, let t = [x(-), (7(-)] be 
another trajectory, and let {tk} be its switching time sequence. Then t is said 
to converge to tp as t ^ oo if there exists an integer N > 0 such that 

Qpitk) = q{ik-i-N) V/c = 0, 1,2,3, 

lim x(tis.\-N -\-j) — 

i — »-+oo 

lim ( i2s_|_jv+7-i-i ^is+N-\-j) — ^7+1 ^3 

2— »-+00 

It can be shown (see [5] for details) that then there exists a sequence {n} C 
(0, +oo) such that Tj+i — Ti ^ T as i ^ oo and, for any A > 0, 

max { \\x{t Ti) — Xp(t)\\ : t € [0, A]} ^ 0 | 

> as z ^ oo. 

mes {t e [0, A] : q{t + n) yf qp(t)} 0 J 

This in particular means that the continuous components x(t) and Xp(9) of the 
trajectories t and tp, respectively, come close not only for the selected time 
instants t = tis+N-i-j and 9 = tj, as was stated in Definition 2. 

Let t converge to tp as t ^ oo. Then it evidently converges to any trajectory 
that is a shift tp’'^(t) := tp(t + t) (r = const > 0) of tp in time. 

Definitions. A periodic trajectory tp = [a^p(-)) 7p(')] Ihe invariant 

domain (10) is said to be locally asymptotically stable in K if for some e > 0, 
any trajectory t = [a;(-),g(-)] such that ||a;(0) — a;p(0)|| < e and q(0) = qp(0) 
converges to tp as t ^ oo. 

Let a periodic trajectory tp be locally asymptotically stable. Then so clearly 
is any trajectory that is a shift tp^^(t) := tp(t + r) (r = const > 0) of tp in 
time. 

Definition 4. A limit cycle is a class ££ of periodic trajectories such that, 
along with any trajectory t, it contains all the trajectories that are shifts of t 
and one of any two trajectories from ££ is a shift of the other. A limit cycle 
Z€ is said to lie in K if any trajectory constituting it lies in K. 

All the periodic trajectories constituting a given cycle evidently have a common 
order, which is called the order of the cycle. 
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Definition 5. A trajectory t is said to converge to a limit cycle Z€ if it con- 
verges to any periodic trajectory constituting ££ . A limit cycle lying in K is 
said to he locally asymptotically stable in K if so is any periodic trajectory from 
it. 

As follows from the foregoing remarks, it suffices to verify any of these properties 
for only one periodic trajectory from the cycle. 

Let us revert to the network in question. The tuple of its parameters 



p := 



> 't/’sigew ’ )}geG,3'eG(g) 



( 11 ) 



(the scaling coefficients Cg are not included since they are regarded as related to 
the switching policy) belongs to the set 



V := 



p : Sg > 0 Vg e G, pg > 0 Vg S , 

P{g, g') > 0 Vg G G, g' e G(g), (1) and (7) hold 



(12) 



In a natural way, this set can be regarded as an analytical manifold of dimension 
Here m is the number of the pairs (g,g') such that g G G and g' G G(g), 
and k is the number of the buffers in the receiving part 91 . 



Theorem 1. Assume that the processing part of the system is not empty and 
Cg := p~^ Vg G 91 . (This scaling means that while dealing with the receiving 
part, the server switches to the buffer with the longest period of being unserved. 
This is true since the beginning of the second service session.) Suppose that 
Assumption 1 and relations (1), (7) hold. Consider the control policy SPl— SP5 
and denote by k the number of the buffers in the receiving part 91 . 

Then the parameter manifold (12) contains a subset E of the zero Lebesgue 
measure such that whenever the tuple (11) of the parameters lies outside E, the 
following statements hold: 



1. There exist limit cycles lying in the invariant domain (10). 

2. Their number equals fc! := 1 x 2 x • • • x A:. 

3. Each of these cycles is locally asymptotically stable in this domain. 

4. Any trajectory lying in it converges to one of the above limit cycles. 



Thus ’’almost all” systems from the class under consideration exhibit a reg- 
ular and predictable behavior. 

From now on, the hypotheses of Theorem 1 are assumed to hold. For given 
tuples of parameters p G V and {cgjg^Ci either the statements 1 — 4 are true 
or the domain (10) contains infinitely many limit cycles, as well as a continuum 
of trajectories that converge to no limit cycle. More precisely, the second case 
occurs if and only if there exists a periodic trajectory for which the largest scaled 
content is attained at several buffers at a moment when the server switches to 
a buffer from the processing part. (Such event never occurs along periodic tra- 
jectories at times when the server switches to a buffer from the receiving part.) 
At this moment, the trajectory splits into a number of continuations. It can be 
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shown that any of them can be chosen periodic. Moreover, they can be chosen 
to have a common period T and so that these trajectories are the same (up 
to a shift of time) during any session of serving the receiving part. The restric- 
tions xi, . . . , Xr of these trajectories on [0, T] can be clearly combined arbitrarily 
Xi(i)’ M(2)> Xi(3)’ ■ • ■ in course of time to form a new trajectory. Corresponding to 
a periodic sequence Xi(i)> Xi(2)j Xi(3)> ■ • ■ is a periodic trajectory whose period is 
multiple of T. Countably many periodic trajectories can evidently be obtained 
so. If the above sequence is not periodic, the trajectory converges to no limit 
cycle. There obviously is a continuum of such trajectories. 

It will follow from the proof of Theorem 1 that its statement is related to the 
policy SPl— SP3 of serving the receiving part of the network much more than to 
that SP4,SP5 of dealing with the processing one. More precisely, this statement 
remains true under various alterations of the second policy. For example, it can 
be replaced by the following one. The server first serves the buffers from S-n, 
then from S-n+i, and so on, up to serving S-i, and then returns to the receiving 
part of the network. Each of the sets S-i is processed on the basis of the Clear- 
the-Largest-Buffer-Level policy. In other words, the server switches when the 
current buffer g G S-i is emptied and to a buffer g G S-i with the largest (over 
S-i) scaled content. Likewise, it starts with a buffer g G S-i having the largest 
value of this content at the moment. The server deals with the layer S-i until it 
becomes empty. (The advantage of this policy is that it excludes multiple passing 
through a buffer within a given session.) 
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Abstract. This paper reports on an on-going project to investigate techniques to 
diagnose complex dynamical systems that are modeled as hybrid systems. In par- 
ticular, we examine continuous systems with embedded supervisory controllers 
that experience abrupt, partial or full failure of component devices. We cast the 
diagnosis problem as a model selection problem. To reduce the space of potential 
models under consideration, we exploit techniques from qualitative reasoning to 
conjecture an initial set of qualitative candidate diagnoses, which induce a smaller 
set of models. We refine these diagnoses using parameter estimation and model 
fitting techniques. As a motivating case study, we have examined the problem of 
diagnosing NASA’s Sprint AERCam, a small spherical robotic camera unit with 
12 thrusters that enable both linear and rotational motion. 

1 Introduction 

The objective of our project has been to investigate how to diagnose hybrid systems 
- complex dynamical systems whose behavior is modeled as a hybrid system. Hybrid 
models comprise both discrete and continuous behavior. They are typically represented 
as a sequence of piecewise continuous behaviors interleaved with discrete transitions 
(e.g., [7]). Each period of continuous behavior represents a so-called mode of the sys- 
tem. For example, in the case of NASA’s Sprint AERCam, modes might include trans- 
late -X-axis, rotate JC-axis, translate. Y-axis, etc. [1]. In the case of an Airbus fly-by- wire 
system, modes might include take-off, landing, climbing, and cruise. Mode transitions 
generally result in changes to the set of equations governing the continuous behavior of 
the system, as well as to the state vector that initializes that behavior in the new mode. 
Discrete transitions that dictate mode switching are modeled by flnite state automata, 
temporal logics, switching functions, or some other transition system, while continuous 
behavior within a mode is modeled by, e.g., ordinary differential equations (ODEs) or 
differential and algebraic equations (DAEs). 

The problem we address in this paper is how to diagnose such hybrid systems. For 
the purposes of this paper, we consider the class of hybrid systems that are continuous 
systems with an embedded supervisory controller, but whose hybrid models contain no 
autonomous jumps, l.e., all nominal transitions between system modes are induced by 
a controller action, none are induced by the system state and model [7]. The class of 
systems we consider can be modeled as a composition of a set of component subsys- 
tems, each of which is itself a hybrid system. We assume that the system operation is 
being tracked by a monitoring and observer system (e.g., [19]) that ensures that the sys- 
tem behavior predicted by the model does not deviate significantly from the observed 
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behavior in normal system operation. When observations occur outside this range, the 
behavior is deemed to be aberrant and diagnosis is initiated. In this paper, we consider 
faults whose onset is abrupt, and which result in partial or complete degradation of 
component behavior. The general problem we wish to address can be stated as follows: 
Given a hybrid model of system behavior, a history of executed controller actions, a his- 
tory of observations, including observations of aberrant behavior relative to the model, 
isolate the fault that is the cause for the aberrant behavior. Diagnosis is done online 
in conjunction with the continued operation of the system. Hence, we divide our diag- 
nosis task into two stages, initial conjecturing of candidate diagnosis and subsequent 
refinement and tracking to select the most likely diagnoses. 

In this paper we conceive the diagnosis problem as a model selection problem. The 
task is to find a mathematical model and associated parameter values that best fit the sys- 
tem data. These models dictate the components of the system that have malfunctioned, 
their mode of failure, the estimated time of failure and any additional parameters that 
further characterize the failure. To address this diagnosis problem, we propose to ex- 
ploit AI techniques for qualitative diagnosis of continuous systems to generate an initial 
set of qualitative candidate diagnoses and associated models, thus drastically reducing 
the number of potential models for our system. This is followed by parameter estima- 
tion and model fitting techniques to select the most likely mode and system parameters 
for candidate models of system behavior, given both past and subsequent observations 
of system behavior and controller actions. The main contributions of the paper are: 1) 
formulation of the hybrid diagnosis problem; 2) the exploitation of techniques for qual- 
itative diagnosis of continuous systems to reduce the diagnosis search space; and 3) the 
use of parameter estimation and data fitting techniques for evaluation and comparison 
of candidate diagnoses. 

In Section 2 we provide a brief description of NASA’s Sprint AERCam, which we 
have used as a motivating example and which we will use to illustrate certain concepts 
in this paper. In Section 3 we present a formal characterization of the class of hybrid 
systems we study and the diagnosis problem they present. In Section 4 we describe our 
approach to hybrid diagnosis and the algorithms we use to achieve hybrid diagnosis. 
The generation of initial candidate qualitative diagnoses is described in Section 4.1, 
and the subsequent quantitative fitting and tracking of candidate diagnoses and their 
models is described in Section 4.2. In the final two sections, we briefiy discuss related 
work and summarize our contributions. 

2 Motivating Example: The AERCam 

We are using NASA’s Sprint AERCam and a simulation of system dynamics and the 
controller written in Hybrid CC (HCC) as a testbed for this work. We describe the 
dynamic model of the AERCam system briefiy, a more detailed description of the model 
and simulation appear in [1]. 

The AERCam is a small spherical robotic camera unit, with 12 thrusters that allow 
both linear and rotational motion (Fig. 1). For the purposes of this model, we assume 
the sphere is uniform, and the fuel that powers the movement is in the center of the 
sphere. The fuel depletes as the thrusters fire. 
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The Body frame of reference 
and the directions of velocities 
(u,v,w) are the components of 
the translation velocity, while 
(p,q,r) are components of the 
angular velocity. 






Three views of the AERCam, showing the thrusters, 
and showing all the thrusters together in the cube 
circumscribing the AERCam. 



Fig. 1. The AERCam axes and thrusters 



The dynamics of the AERCam are described in the AERCam body frame of refer- 
ence. The translation velocity of this frame with respect to the shuttle inertial frame of 
reference is 0. However, its orientation is the same as the orientation of the AERCam, 
thus its orientation with respect to the shuttle reference frame changes as the AERCam 
rotates (i.e., it is not an inertial frame). The twelve thrusters are aligned so that there 
are four along each major axis in the AERCam body frame. For modeling purposes, 
we assume the positions of the thrusters are on the centers of the edges of a cube cir- 
cumscribing the AERCam. Thus, for example, thrusters Tx,T2,T^,Ti are parallel to 
the x-axis and are used for translation along the a;-axis or rotation around the y-axis. 
I.e., firing thrusters Ti and T2 results in translation along the positive a;-axis, and firing 
thrusters Ti and T4 results in a negative rotation around the y-axis. AERCam operations 
are simplified by limiting them to either translation or rotation. Thrusters are either on 
or off, therefore, the control actions are discrete. In a normal mode of operation, only 
two thrusters are on at any time. 



2.1 AERCam Dynamics 

A simplified model of the AERCam dynamics based on Newtonian laws is derived us- 
ing an inertial frame of reference fixed to the space shuttle. The AERCam position in 

this frame is defined as the triple (x, y, z). Let V be the velocity in the AERCam body 
frame, with its vector components given by (u, v, w). The frame rotates with respect 
to the inertial reference frame with velocity oj = {p, q, r), the angular velocity of the 
AERCam. The rotating body frame implies an additional Coriolis force acting upon the 
AERCam. We assume uniform rotational velocity since in the normal mode of opera- 
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tion, the AERCam does not translate and rotate at the same time [2, pg. 130]. Similar 
equations can be derived for the rotational dynamics [1], 

d(m V)/dt =F — 2m(V x u>) Newton’s Law 
V dm/dt + mdiy) / dt =F — 2m(w x V) 

The resultant equation for each coordinate: 

du/dt = F^jm — 2{qw — vr) — {ulm) * dm/dt 
dv/dt — Fy /m — 2(ru — pw) — (v/m) * dm/dt 
dw/dt = F^/m — 2(pv — qu) — (w/m) * dm/dt 



2.2 Position Control Mode of the AERCam 

In the position control mode, the AERCam is directed to go to a specified position and 
point the camera in a particular direction. Assume the AERCam is at position A and 
directed to go to position B. In the first phase, the AERCam rotates to get one set of 
thrusters pointed towards B. These are then fired, and the AERCam cruises towards B. 
Upon reaching a position close to B, it fires thrusters to converge to B, and then rotates 
to point the camera in the desired direction. 

To facilitate the illustration of the diagnosis problem, we use a simple trapezoidal 
controller, which we explain in two dimensions. Suppose the task is to travel along 
the a;-axis for some distance, then along the y-axis. Such manoeuvres are needed for 
navigating in the space shuttle. In order to do this, the AERCam fires its x thrusters 
for some time. Upon reaching the desired velocity, these are switched off. When the 
AERCam has reached a position close to the desired x position, the reverse thrusters are 
switched on, and the AERCam is brought to a halt — the velocity graph is a trapezium. 
The process is analogous for the y direction. 

3 Problem Formulation 

In this section we provide our formulation of the hybrid diagnosis problem. 

Definition 1 (Hybrid System). A hybrid system is a 5-tuple (AI, A, tF, S, 4>), where 

- M, finite set of system modes (/ri, ... ,yk)- 

-AC i?", continuous state variables. x{t) is the continuous behavior at time t. 

- F , finite set of functions . . . , and associated parameter values 6 such 

that for each mode, pti, {t, 6, x{t)) : R x R x X ^ X defines the continuous 
behavior of the system in pi/ 

- X, finite set of actions (cti, ... , u/), which transition the system between modes. 

- (f), transition function which maps an action, mode and system state vector into a 
new mode and initial state vector, i.e., (p: SxMxX^MxX. 

To define the hybrid diagnosis problem, we augment Definition 1 as follows. 

* Parameter value ranges may be associated with 6. 
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Definition 2 (Diagnosable Hybrid System). A diagnosable hybrid system, 
(At, X, A, E, (j), COMPS) is a hybrid system comprised of m potentially malfunc- 
tioning components COMPS = (ci, . . . , Cm) where 

- For each /i e Al, /r includes a designation of whether each a e COMPS is 
operating normally, or abnormally, i.e., {^)ab{ci). 

- We assume that transitions to fault modes are achieved by exogenous actions. 
Hence, E = EcLI Se, where 

• 27c is a finite set of controller actions, and 

• 27e is a finite set of exogenous actions. 

- A, the controller action history, the sequence of time-indexed controller actions 
performed. 

- Xobs C X, continuous state variables that are observable. Xobs{t) is the observa- 
tions at time t. 

- O, the observation history, the sequence of time-indexed observations. 

For notational convenience, fip denotes a faulty mode, i.e., a mode for which at least 
one Ci e COMPS is ab{ci) in fip. Op denotes the parameters associated with 

In the case of the AERCam example, the potentially malfunctioning components are 
the 12 thrusters, and a mode ^ includes the behavior mode (e.g., translate-x, translate- 
y, rotate-x, etc.) and {-)ab{Ti), z = 1, . . . , 12, for each thruster. The continuous state 
vector includes the x, y, z position of the AERCam, velocity and acceleration. The 
parameter values, 9 associated with each are the percentage degradation of each of 
the thrusters. 

Definitions (Model). A model. Mod of a diagnosable hybrid systems is a time- 
indexed mode sequence and associated parameter values ([^i, . . . , Hm], [9i, • ■ . , 0m])- 

Notice that each model of the system, (/x, 6) induces a corresponding time-indexed 
piecewise continuous sequence of functions , • ■ • , dictating system behavior. 

In this paper we make several simplifying assumptions regarding our diagnosis task. 
In particular, we make a single-time fault assumption. We assume that our systems do 
not experience multiple sequential faults. Further, we assume that faults are abrupt, 
resulting in partial or full degradation of component behavior. We cast the hybrid diag- 
nosis task as the problem of finding the most likely model for the observation history, 
P{Mod I O). I.e, the sequence of modes and parameter values (/x, 0) that best fit the 
observations over time. Under normal operation, the model of the system M odnormai is 
fully dictated by the sequence of controller actions A and the nominal parameter values, 
9. Once again, we assume that the system operation is being tracked by a monitoring and 
observer system (e.g., [ 1 9]) that ensures that the system behavior predicted by the model 
does not deviate significantly from the observed behavior in normal system operation. 
When observations occur outside this range, the behavior is deemed to be aberrant and 
diagnosis is initiated. Given a diagnosable hybrid system (Al, X, A, 27, (j), COMPS), 
a controller action history, A and a history of observations, O which includes observa- 
tions of aberrant behavior, the hybrid diagnosis task is to determine what components 
are faulty, what fault mode caused the aberrant behavior, when it occurred, and what the 
values of the parameters associated with the fault mode are. In the AERCam system, a 
diagnosis might be that thruster Ti experienced a blockage fault of 50%, at time ti. 
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Once Modnormai has been rejected, we must find a new most likely model from 
among the potentially exponential (in COMPS) number of mode sequences, occurring 
within a large but bounded time range. We propose to exploit previous research on 
temporal causal graphs for qualitative diagnosis of continuous systems [ 1 8], to compute 
a set of candidate qualitative diagnoses that are consistent with our system, in order to 
identify a preliminary subset of candidate models, whose likelihood can be estimated. 

Definition 4 (D-tnple). A D-tuple is a 4-tuple {C, fiF,tF,0p), where fXF is a fault 
mode, tF is the time the fault mode commenced. Of is the parameter values associated 
with the fault mode behavior, and C is the set of failed (abnormal) components m^F- 

Definition 5 (Candidate Qnalitative Diagnosis). Given a diagnosable hybrid system 
with model Mod= (/x, 9) an action history A, and a history of observations, O which 
includes observations of aberrant behavior. D-tuple {C,fiF,tp, O f) isa candidate qual- 
itative diagnosis iff there exists a range of parameter values Of = [0i,0y], and time 
range tF = [ti, such that the occurrence of fault mode with parameter values Of 
in time range tp is consistent with O, A and Mod. 

Hence, a candidate qualitative diagnosis stipulates a fault mode, including one or 
more faulty components. It also stipulates a lower and upper bound, [ti,tu], on the time 
the fault mode occurred. This range generally corresponds to the start times of the con- 
troller induced modes preceding and following the fault, or up to the point the fault was 
detected. This candidate diagnosis induces an associated candidate model, Mode = 
{[di,- ■ ■ ,6»J) corresponding to 

Mod with the fault mode /xf and Of inserted at tF- Every subsequent mode, m+i, ■ ■ ■ , 

has ab{ci), Ci G C enforced, and every subsequent set of parameters has the param- 
eters associated with faulty components C enforced. Computing candidate qualitative 
diagnoses is discussed in Section 4. 1 . 

Since each candidate qualitative diagnosis only conjectured ranges for the time of 
the fault mode. If and parameter values associated with the fault mode. Of, the asso- 
ciated candidate models are underconstrained. In Section 4.2, we discuss methods for 
estimating unique values for If and Of and for estimating a posterior probability for 
each of the candidate models. Mode, given O. 

Definition 6 (Candidate Diagnosis). Given a diagnosable hybrid system, a history of 
controller actions A, and a history of observations O, D-tuple {C, ^ifAfiOf) with 
associated model Mode is a candidate diagnosis for the hybrid system, iff P{Mode \ 
O) > a, for defined threshold value a e [0, 1]. 

4 Diagnosing Hybrid Systems 

In this section we discuss one method for computing hybrid diagnoses. In Section 4.1 
we discuss a technique for generating candidate qualitative diagnoses, and their associ- 
ated candidate models. In Section 4.2 we discuss techniques for model fitting and for 
model (and hence diagnosis) comparison. In particular we discuss techniques for esti- 
mating the parameters of the candidate models, and the likelihood of the models, and for 
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continued monitoring and refinement of the candidate models as the system continues 
to operate and observations continue to be made. 

We illustrate these techniques with the following simple AERCam example. Con- 
sider the scenario depicted in Fig. 2. In the first accelerate phase, the AERCam is being 
powered by thrusters T1 and T2. Assume that at some point in this phase, a sudden leak 
in the T2 thruster causes an abrupt change in its output. As a consequence, the AER- 
Cam starts veering to the right of the desired trajectory, as illustrated by the left-most 
dotted lines in Fig. 2. (The other dotted lines represent other potential candidate diag- 
noses consistent with the point of detection of the failure.) Soon after this occurs, the 
supervisory controller commands the AERCam to turn off Thrusters T1 and T2 with 
the objective of getting the AERCam to cruise in a straight line. In the faulty situation, 
the AERCam has some residual angular velocity about the z-axis, so it continues to 
rotate in the cruise mode. Then the controller turns on thrusters T3 and T4, to decel- 
erate the AERCam with the objective of bringing it to a halt. Again, this objective is 
not entirely achieved in the the faulty situation. Next, thrusters T5 and T6 are switched 
on, to move the AERCam in the y direction. However, since the AERCam is not in the 
desired orientation after the failure, the position error due to faulty thruster T2 accumu- 
lates causing a greater and greater deviation from the desired trajectory of the system. 
The position of the AERCam is being continuously sensed, filtered for noise and mon- 
itored. At some point within the y translation the trajectory exceeds the error bound, 
i.e., P{Modnormai < «) and is fiagged by the monitoring system as aberrant relative 
to Modnormai- At this point, the diagnosis task begins. 



Desired 

Trajectory 



Error Bounds 



Accelerate 


Cruise 


Decel. 













y-axis 

"-j 

II ' 



Fault 

detected 



Possible 
„ Fault , 



Fig. 2. Possible fault trajectories of AERCam ( simplified for illustration purposes). 



4.1 Qualitative Candidate Generation 

Given the current system model Mod = (/x, 6) (commonly Modnormai), a history of 
controller actions A, and a history of observations O including one or more observa- 
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Fig. 3. A subset of the temporal causal graph showing the relations between Thrusters T1 — T8 
and the x and y positions of the AERCam. 



tions of aberrant behavior, we wish to generate a set of candidate qualitative diagnoses 
{C, fiF,tF,dF), and associated candidate models as described in Definition 5. To do 
so, we extend techniques for generating qualitative diagnoses of continuous dynamic 
systems to deal with hybrid systems with multiple modes. The model and propagation 
mechanism, as applied to continuous systems diagnosis, is described in [18]. 

In the case of our AERCam example, the action history A is [(on(Tl), on(T2)), 
(off(ri), off(T2)), (on(T3), on(T4)), (off(T3), off(T4), on(T5), on(T6)), (off(T5), 
off(T6))]; the model, Modnormai is the time-indexed sequence [{accelerate-X, 
^a6(n — T12), 0), {cruise JF, -^ab{Tl — T12), 9) , {decelerate jf , ^ah{Tl — T\2), 9), 
{acceleratejy, ^ab{Tl — T12), 9),{cruisejj, ^ab{Tl — T12),9)\, where 0 is a vector 
of length 12 all of whose entries are 0 (percent degradation in thrusters). 

To generate candidate qualitative diagnoses we construct an abstract model of the 
dynamic system behavior, Modnormai as a temporal causal graph. A part of the tem- 
poral causal graph for the AERCam dynamics is shown in Fig. 3. The graph expresses 
directed cause-effect relations between component parameters and the system state vari- 
ables. Links between variables are labeled as: (i) -Fl, implying direct proportionality, 
(ii) —1, implying inverse proportionality, and (iii) /, implying an integrating relation. 
An integrating relation introduces a temporal delay in that a change on the cause side of 
the relation affects the derivative of the variable on the effect side. This adds temporal 
characteristics to the relations between variables. Some edges are labeled by variables, 
implying the sign of the variable in the particular situation defines the nature of the rela- 
tionship. The candidate generation algorithm is invoked for every initial instance of an 
aberrant observation. The aberrant observation plus the controller action history A are 
input to a backward propagation algorithm that operates on the temporal causal graph. 
The algorithm operates backwards from the last mode in the mode sequence of Mod: 
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Step 1 For the current mode, extract the corresponding temporal causal graph model, 
and apply the Identify Possible Faults algorithm. Details of this algorithm are presented 
in [18], but the key aspect of this algorithm is to propagate the aberrant observation ex- 
pressed as a ± value, backward depth-first through the graph. For example, given that 
the y— position of the AERCam has deviated — (i.e., below normal), backward prop- 
agation implies d{y)/dt is — , and so on, till we get Tf and Tg", implying thrusters 
T5 and T6 are possibly faulty with decreased thrust performance. Propagation along a 
path can terminate if conflicting assignments are made to a node. The goal is to system- 
atically propagate observed discrepancies backward to identify all possible candidate 
hypotheses that are consistent with the observations. In our example, the component 
parameters, COMPS = {Tl, . . . , T12} form the space of candidate faults. 

Step 2 Repeat Step 1 for every mode in the mode sequence, to . The system model 
needs to be substituted as the algorithm traverses the mode sequence backwards. There- 
fore, back propagation will be performed on a different temporal causal graph for each 
mode in the controller history^. 

The output of this step is a set of qualitative diagnoses {C, fip, tp, dp), each with 
an associated candidate model, as described in Section 3. Returning to our AERCam 
example, three qualitative candidate diagnoses are generated. The first candidate diag- 
nosis is that T2 failed in the x acceleration phase. The time of the fault mode transition 
is [fi, ^ 2 ], and the parameters associated with the failure - the percentage degradation 
of the component is in the range [0, 100]. So the first candidate qualitative diagnosis 
is (T2, {accelerates, ab{T2), ^ab{Tl, T3 — T\2),9p), [^ 1 ,^ 2 ], [0, 100]). The candi- 
date model simply has {accelerates, ab{T2), ^ab{Tl),^ab{T‘i—T12)) inserted after 
the mode {accelerates, ~^ab{Tl — T12)), and ab{T2) enforced in every subsequent 
mode. The second candidate qualitative diagnosis is that T4 failed in the deceleration 
phase of x translation, i.e., (T4, {decelerates, ab{T4), ^o6(Tl — T3, Tb — T\2),9p), 
[ts, tf, [0, 100]). The third candidate is that T6 failed during y acceleration, i.e., (T6, 
{acceleratejy, ab{T6), ^ab{Tl — T5, T7 — Tl2),9p), [f 4 , to], [0, 100]), where fu is 
the time of detection of the aberrant behavior. In each case 9p is a vector of length 12 
with every entry equal to 0 (percentage degradation), except the entries corresponding 
to the faulty thrusters, C which will have the range [0, 100]. 

4.2 Model Fitting and Comparison 

Given the candidate qualitative diagnoses and their associated candidate models, the 
next phase of the diagnosis process is quantitative refinement of the qualitative can- 
didate diagnoses and their associated models through parameter estimation and data 
fitting, followed by tracking of the fit of subsequent observations to the candidate mod- 
els. The goal is to at least provide a probabilistic ranking of the plausible candidates, if 
not a unique model (and hence diagnosis). 

As observed in the previous section, the model associated with the candidate qualita- 
tive diagnosis. Mode is underconstrained. Both the time of the fault mode occurrence, 
tp and the parameters associated with the faulty behavior are represented as ranges 

^ We may cut off back-propagation along the mode sequence beyond a time limit. 
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and must be estimated. Further, the candidate qualitative diagnoses were generated from 
initial observations of aberrant behavior, and their consistency can be further evaluated 
by monitoring the qualitative transients associated with each candidate. The refinement 
process is performed by a set of trackers [21], one for each candidate diagnosis and 
associated model. Each tracker comprises both a qualitative transient analysis compo- 
nent and a quantitative model estimation, component. The two components operate in 
parallel as described below. 

Qualitative Transient Analysis 

The qualitative transient analysis component performs a further qualitative analysis of 
the consistency of candidate qualitative diagnoses based on monitoring of higher-order 
transients whose manifestation is seen over a longer period of time. If the transients 
of a candidate qualitative diagnosis do not remain consistent with subsequent observa- 
tions, the candidate diagnosis will be eliminated and the model estimation component 
informed. The technique we employ is derived from techniques for qualitative monitor- 
ing of continuous systems. Details of the algorithm appear in [18]. 

Model Estimation 

The purpose of the model estimation component is to perform quantitative model fit- 
ting, i.e., to provide a quantitative estimate of the parameters of the models and to assign 
a probability to each of the candidate models (and hence candidate diagnoses), given 
the noisy observed data. In particular, given a candidate model. Mode the model es- 
timation component uses parameter estimation techniques to estimate both the time at 
which the failure occurred, t p , and the value for the parameters, , associated with the 
conjectured failure mode. In this paper we discuss two alternate approaches to our time 
and parameter estimation problem. The first approach is based on Expectation Maxi- 
mization (EM) (e.g., [8]), an iterative technique that converges to an optimal value for 
tp and 6p simultaneously. The second approach we consider employs General Likeli- 
hood Ratio (GLR) techniques (e.g., [5]) to estimate the time of failure tp, and then uses 
the observations obtained after the failure to estimate the fault parameters, , by a least 
squares method. As described in Section 3, the outcome of both approaches is a unique 
value for tp and 6p and a measure of the likelihood of Mode given the observations. 
The proposed approaches to model fitting have trade-offs and we are currently assess- 
ing the efficacy of these and other alternative approaches through experimentation. 

EM-Based Approach The Expectation Maximization (EM) algorithm (e.g., [8]) pro- 
vides a technique for finding the maximum-likelihood estimate of the parameters of an 
underlying distribution from a given set of data, when that data is incomplete or has 
missing values. The parameter estimation problem we address in this paper is a vari- 
ant of the motion segmentation problem described in [24]. Here, we define the basic 
algorithm and the intuition behind our approach. (See [8] for more details.) 

The time of failure, tp = [ti, ty] of our candidate qualitative diagnosisdictates the 
mode in which the failure is conjectured to have occurred. Let us call this mode /uf 
T he behavior of our hybrid system in mode iii is described by the continuous function 
with known parameters 9i. At some (to be estimated) time point tp within the 
predicted time period of Hi, we have conjectured that the system experienced a fault 
which transitions it into mode yLp. The behavior of our hybrid system in mode fxp is 
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described by the continuous function f^p , with unknown parameters, 9p. We also have 
a set of data points O = [xobs{ti), ■ ■ ■ ,Xobs{tu)\ C O, which either reflect the behavior 
of the system under /^ . or under . 

Given all this information, our task is to And 1) values for parameters 9p, and 2) an 
assignment of the data points O to either Hi or fxp so that we maximize the fit of the 
data to the two functions. The assignment of data points will in turn tell us the value 
of tp. EM provides an iterative algorithm which converges to provide a maximum- 
likelihood estimate for given O , i.e., roughly we are calculating the likelihood of 9, 
L{9) = P{0' I 9p,Modc). 

The basic EM algorithm comprises two steps: an Expectation Step (E Step), and a 
Maximization Step (M Step) [24]: 

• Select an initial (random) value for 9p. 

• Iterate until convergence: 

- E Step: assign data points to either /^. (9i) or (9p), which ever fits it best. 

- M Step: re-estimate 9p using the data points assigned to {Of)- 

The assignment of data points to and fip provides an estimate for fp. We may 
exploit the fact that the assignment of data points is temporally correlated with all points 
before f/ belonging to and all points after belonging to fif. We may also exploit 
the fact that data points at the beginning of the interval will belong to fii, while those 
at the end will belong to fip. These task-specific qualities help our algorithm converge 
more quickly. 

EM provides a rich algorithm for maximum-likelihood parameter estimation when 
we don’t know the value of In some hybrid diagnosis applications, depending upon 
the sensors in our system, and the level of noise in the sensors, we may be able to de- 
velop monitoring techniques that will help isolate a reasonable value for tp, minimizing 
the need for iteration in EM. In such cases, an alternative to the EM-based approach is 
to first estimate tp using the Generalized Likelihood Ratio (GLR) method [5], followed 
by parameter estimation of 

GLR + Least Squares Approach Here, we divide the parameter estimation problem 
into two parts: (i) estimate the time of failure, tp, using the Generalized Likelihood 
Ratio (GLR) method, and (ii) apply a standard least squares method for parameter esti- 
mation. The intuition is that solving the problem in two parts simplifies the estimation 
process, and very likely mitigates the numerical convergence problems that arise in 
dealing with complex higher-order models. 

The GLR method for detecting abrupt changes in continuous signals is described 
in [5]. We have applied it to fault transients analysis in complex fluid thermal systems 
[16]. Here we provide an overview of the method for the single parameter case. Assume 
that the signal under scrutiny is a time-indexed sequence of random variables y{k), with 
probability density function, PBi{y) in desired mode pi, and P 0 p{y) in fault mode pp. 
y is either contained in Xobs or computed from Xobs- We assume that a fault causes an 
abrupt change in y{k). In the case of the AERCam, y captures the difference between 
the observed and expected values of the, e.g., acceleration, as predicted by the model. 
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The central quantity in the change detection algorithm is the cumulative sum of the 
log-likelihood ratio for a window of observations between times m and n, 



SKOf) = 



k—m 



P9p{y{k)) 

PBiiyik)) ' 



Again, this ratio is a function of two unknowns: tp and 9f- The common statistical 
solution is to use maximum likelihood estimates for these two parameters, resulting in 
a double maximization: 



= max sup5^(6If). 

l<m<n 

If we assume that probability density functions, pe^ (y) and pgp (y) are Gaussian, 
then (/„ reduces to: 

1 1 

max — - 

Zaf i<m<n n — m + 1 

where uji and af are the mean and variance forp^i (y), respectively. 

When processing a sequence of samples, the point of abrupt change, fp, is computed 
from min{n : > h}, where h is an appropriately defined threshold. Hence, the 

smaller the value of h, the more sensitive the function to change, and unfortunately to 
false alarms, so h must be set carefully. 

Once t F is estimated, data points observed after ^f , are used to estimate the parame- 
ter, 0F for a hypothesized fault using regression techniques. In the case of the AERCam, 
the position vector of the AERCam is modeled as a set of quadratic functions in terms 
of the thruster force. These functions contain one unknown, dp, the parameter that cor- 
responds to the degree of degradation in the faulty thruster. The least squares estimate 
for 0F is computed, and the the measure of fit of the candidate model to the observed 
data used to estimated the probability of the candidate model (and hence, diagnosis). 

Model Comparison 

From the model estimation component, each tracker computes the likelihood of its 
model Mode, and hence of the associated candidate diagnosis (C, pF, if, 6 *f), as a 
measure of fit of the observations to the model. As new data Xobs{t) are observed, 9p 
and tp, are adjusted and P{Modc \ Xobs{t)) computed. If the likelihood of Mode 
falls below a predefined acceptable likelihood threshold, a, then its tracker is termi- 
nated, and the associated candidate diagnosis {C, pp,tp,9p) removed from the list of 
candidate diagnoses. Tracking terminates when a unique diagnosis is obtained, or when 
the diagnoses are sufficiently discriminated to determine suitable controller actions. 

5 Related Work 

The specific problem of diagnosing hybrid systems has received little attention to date, 
although there is much related work. Within the AI community, there has been a great 



_k—m 
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deal of research on diagnosing static systems (e.g., [14]), while much less on diag- 
nosing discrete dynamical systems (e.g., [17,25]), and qualitative representations of 
continuous systems (e.g.,. [18]). Within the FDI community, the largest proportion of 
research has focused on diagnosing continuous systems (e.g., [13,11]). The most com- 
mon model-based approaches use observer schemes(e.g., [12,20]), where the goal is to 
design residual generators based on observed discrepancies, such that individual resid- 
uals are sensitive to a particular subset of faults. There is also complementary work by 
Basseville [4], using model-based statistical processing techniques for early fault de- 
tection and residual identification. [18] perform residual generation and analysis task in 
a qualitative framework to address some of the computational issues that arise in han- 
dling the complex dynamics that occur in fault transients, with some preliminary work 
on building multiple observers for hybrid systems [19]. Diagnosis of discrete-event sys- 
tems has also been studied within the FDI community (e.g, [22,15]). Fabre et al. [10] 
have employed stochastic Petri nets based on a Hidden Markov Model probabilistic 
scheme for alarm analysis. Unfortunately, it is not clear how to systematically derive 
such representations from the physical system models that we work with. 

6 Summary 

In this paper we addressed the problem of diagnosing hybrid systems. The main con- 
tributions of the paper are 1) formulation of the hybrid diagnosis problem as model 
selection; 2) the exploitation of techniques for qualitative diagnosis of continuous sys- 
tems to reduce the diagnosis search space; and 3) the use of parameter estimation and 
data fitting techniques for evaluation and comparison of candidate diagnoses. This work 
continues with experimental analysis of the proposed techniques, and a more formal 
characterization of our approach in terms of Bayesian model selection. 
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Abstract. We define a new class of hybrid automata for which reach- 
ability is decidable — a proper superclass of the initialized rectangular 
hybrid automata — by taking parallel compositions of simple components. 
Attempting to generalize, we encounter timed automata with algebraic 
constants. We show that reachability is undecidable for these algebraic 
timed automata by simulating two-counter Minsky machines. Modifying 
the construction to apply to parametric timed automata, we reprove the 
undecidability of the emptiness problem, and then distinguish the dense 
and discrete-time cases with a new result. The algorithmic complexity — 
both classical and parametric — of one-clock parametric timed automata 
is also examined. We finish with a table of computability-theoretic com- 
plexity results, including that the existence of a Zeno run is -complete 
for semi-linear hybrid automata; it is too complex to be expressed in 
first-order arithmetic. 



1 Introduction 

Though the bulk of this paper will be given over to undecidability results, our 
initial motivation is the extension, even by a small amount, of the class of hybrid 
automata for which reachability is known to be decidable. It has been suggested 
that it is the coupling of continuous variables which leads to undecidability [7]. 
Parallel composition couples only the discrete dynamics of its components. Thus, 
arguing informally, if we consider parallel compositions of hybrid automata which 
obey a sufficient decoupling between discrete and continuous dynamics, then we 
should be able to circumvent undecidability. We will bring this simple idea to a 
simple fruition in Sect. 2, but first we must dispose of the preliminaries. 

1.1 Hybrid Automata 

A hybrid system is a physical system which combines discrete and continuous 
dynamics. Hybrid automata are intended as formal mathematical models of such 
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systems. The following definition is provided to fix notation for the duration of 
this paper. Though no standard definition exists, this one is not unusual. Note 
that the continuous dynamical behavior is expressed by a (non-deterministic) 
semi-flow, not by vector fields as is more common. 

Definition 1. A hybrid automaton A is a tuple {Q,S, X,l,S,s,U,TZ,^) such 
that: 

• [discrete states] Q is a finite set 

• [edges] £ is a finite set 

• [plant states] X is any set (usually taken to be a manifold) 

• [invariant set] J C Q x X 

• [initial set] S C Q x X 

• [source map] s : £ — > Q 

• [destination map] t) : £ — > Q 

• [reset relation] TZ C X x S x X 

• [semi-flow] ^ Qx X x R>o — > V{X) such that for all {q,x) G Q x A: 

1. d)(( 7 , X, 0) = {x} 

2. Vti,t2GR>o +t2) = U ye<c(9.a:,ii) 



The components of a hybrid automaton A are written with A as a superscript, 
as in Q-^, and d>'^. The superscript may be omitted when the automaton 
is clear from context. Tq denotes the invariant set in discrete state q and is 
taken to be a subset of X. Similarly, Sq, TZe and d)g are given their expected 
interpretations as subsets of X, X^ and X x M>o x A, respectively. Finally, by 
the guard of and edge e€S we refer to the support of the reset relation TZe- 



Definition 2. A run of a hybrid automaton A is a sequence {qo, xq, /o> to, yo, 
eo, qi, xi, fi, ti, yi, ei, . . . , e„_i, g„, Xn, fn, tn, Vn) such that for all 0 < t < n: 



• qt€ Q 

• Xl,y^ G A 

• {qo,xo) G S 

• ti G M>o 



. fl : [0,t,] ^ A 

• /j(0) = Xi and fflti) = y^ 

• VtG [0,t*) /j(t) G I 

• Vs,tG[0,t*] s <t — > fflt) G ^{qi, fi{s),t - s) 



and for all 0 < t < n: 

• €i G E 

• s(ei) = q^ and d{ei) = qi+i 

• (yi,ei,Xi+i) G TZ. 

In Sect. 5 we will generalize the notion of run both by allowing the final time 
interval to be infinite and by allowing infinite sequences of transitions. Until 
then, finite runs will be more convenient. 



Definition 3. The semi-linear (resp. semi- algebraic) subsets of M” are formed 
by taking boolean combinations of sets defined by linear (resp. algebraic) equal- 
ities and inequalities with rational coefficients. 
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Definition 4. By semi-linear hybrid automata (SLHA) we mean that elusive 
class of automata which has been variously known as polyhedral and — to the 
consternation of control theorists — as linear. A is an n-dimensional SLHA if: 

• = M” for some n 

• for every q e Q-^ and e G the projected components 1^, S^, TZ-^ and 

are semi- linear subsets of M”, M”, and respectively. 

Semi- algebraic hybrid automata are defined analogously. 

1.2 Annotated Hybrid Automata 

It will be convenient to add a layer of abstraction to our hybrid automata. An 
annotation associates to each edge an event and to each discrete state a nonempty 
set of possible conditions. These annotations do not affect the behavior of the 
automaton but will be used when we define the timed language of an automaton 
and when we define the operation of parallel composition. 

Definition 5. An annotated hybrid automaton A is a hybrid automaton with 
four additional components (S,T, e, c): 

• [events] S is a finite set 

• [conditions] T is a finite set 

• [event assignment] e : f — > S 

• [condition assignment] c : Q — > V{T) such that VgG Q c{q) yf 0. 

Definition 6. To each run (go, xq, fo, to, Vo, cq, qi, xi, fi, h, yi, ei, ... , 
e„_i, qn, Xn, fn, tnj Pn) of an annotated hybrid automaton A, we associate an 
annotated run (cq, to, xg, ci, ti, ui, . . . , c„, t„) such that: 

• for all 0 < t < n, Ci G c(qi) 

• for all 0 < t < n, t>i = e(ci). 

The timed language L{A) of an annotated hybrid automaton A is set of all 
annotated runs of A. 

The following equivalence relation will be important. 

Definition 7. We say that the annotated hybrid automata A and B are lan- 
guage equivalent iff: 

• = S® 

• C{A) = C{B). 

We denote language equivalence by A ~ie B. 

Remark 1. Invoking symmetry, one might expect the requirement that T"^ = T® 
in the definition of language equivalence. We disclude this requirement because it 
is unnecessary, though it would not falsify the results that follow. The interested 
reader should note in Sect. 1.3 that the set T does not play a very important 
role in parallel composition, while S is crucial. 
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By the reachability problem for an annotated hybrid automaton A, we mean 
the problem of determining which conditions c G occur on some annotated 
run. This ensures that language equivalent hybrid automata have equivalent 
reachability problems. Of course, the reachability of a discrete state can be 
detected with a suitable annotation and we may suppress explicit mention of 
annotations when discussing reachability. We say that the reachability problem 
is decidable for a class 1 C if there is an algorithm which uniformly solves the 
reachability problem for every member of 1 C. 



1.3 Parallel Composition 

Given two annotated hybrid automata we define a product automaton called the 
parallel composition. Conceptually, a run of the parallel composition is comprised 
of simultaneous runs of the component automata which are independent except 
that: 

• They must synchronize on shared events. 

• The only product states that are permitted are those for which the restric- 
tions on conditions are jointly satisfiable. 



Definition 8. We define the parallel composition .4 |1 ,8 of the annotated hy- 
brid automata A and B in two stages. First, we define a synchronized product 
automaton A® B such that: 

• Q=Q^xQ^ 



• £ = U £2 U £3 where: 

£1 = {(ei,92) G X I e-^(ei) i S»} 

^2 = {(<71,62) G X I e»(e2) i S-^} 

£3 = {(61,62) G X I e-^(ei) = e®(c2) 

• X = xX^ 


















I = {((91,52), {xi,X2)) G Q X T I (91, Xi) G A (92,3^2) G X®} 
S = {((91,92), (a;i,a;2)) C Qx X \ {qi,xi) G 5-^ A (92,3^2) G 5®} 



5(61,62) = 



^(61,62) = 



(s-^(ci),C2) 


if 62 GQ® 


(C1,S®(C2)) 


if 61 G 


(S-^(C1),0®(C2)) 


otherwise 


(0-^(ci),C2) 


if 62 GQ® 


(C1,C)®(C2)) 


if 61 G Q-^ 


(0-^(ci),C)«(c2)) 


otherwise 



TZ = {{{xi,X2),{ci,C2),{yi,y2)) C X x S x X \ 

((ci e Q-^ A xi = yi) or (ci G £-^ A (xi,ci,yi) G and 

((c2 G Q® A X2 = 92) or (c2 G X® A {x2, 62, 92) G 7^®))} 
^•((91, 92), (aii,a:2),r) = <P-^{qi,xi,r) x $^(92, 3:2, r). 



A® B is annotated as follows: 

• S = U S® 

• r = r-^ n r® 

( 1 _ / ^"^(ci) ifeiGX-^ 

2(61,62) |^e®(c2) otherwise 

• 2(91,92) = c-^(9i) n 2^(92). 




300 



J.S. Miller 



The second stage in the formation of ^ |1 S is to discard all discrete states 
q G such that = 0. This ensures that A\\ B is an annotated 

hybrid automaton and completes the construction. 



Remark 2. Parallel composition is commutative and associative (up to isomor- 
phism). Therefore we can, and will, refer to the parallel composition of several 
annotated hybrid automata without fear of ambiguity. 

The concept of parallel composition defined here is nowise new. Conditions 
are just an alternative to the propositional constraints that commonly arise in 
the temporal logic literature. The novelty is not in our definition, but in the use 
we will make of parallel composition — to define a new class of hybrid automata 
for which the reachability problem is decidable. The following simple relationship 
between language equivalence and parallel composition will be a key ingredient; 
it will allow us to do reductions component- wise. 

Lemma 1. If A ~ie A' and B ~ie B' then A |1 B ~ie A! |1 B' . 

2 A New Decidable Class 

Definition 9. If /C is a class of hybrid automata, then the parallel closure is 
the class of all parallel compositions of all annotations of the elements from 1C. 



Definition 10. 

Clock Components: 

Let C be the class of 1-dimensional SLHA such that d)(( 7 , t,x) = x + t, the plant 
state is zero in all initial states, and each edge satisfies either: 

(a) zero reset 
or (b) identity reset 

Rectangular Components: 

Let TZ be the class of 1-dimensional SLHA such that d)(( 7 , t,x) = x + tig, where 
Iq is an interval for each q, and such that each edge satisfies either: 

(a) constant set-valued reset map 
or (b) identity reset and 

source and destination have the same flow 

Deterministic Components: 

Let T> be the class of SLHA with deterministic flows and finite initial set such 
that each edge satisfies either: 

(a) constant (single-valued) reset map 
or (b) identity reset and 

source and destination have the same flow 




Decidability and Complexity Results 301 



Nondeterministic Components: 

Let Af be the class of SLHA such that each edge satisfies either: 

(a) constant set-valued reset map 
or (b) identity reset and 
trivial guard and 

source and destination have the same flow and invariant set 

The reader is probably already familiar with and 7^^^, though our pre- 
sentation is somewhat unusual. They are, respectively, timed automata [1] and 
initialized rectangular hybrid automata [12,7]. Both of these classes are known 
to have decidable reachability problems. 

Lemma 2. 

1. If A G TZ then every annotation of A is language equivalent to a two clock 
timed automaton. 

2. If A € T> U Af then every annotation of A is language equivalent to an 
annotation of a clock component. 

Part (1) is contained in [12] while Part (2) offers no real difficulty. Combining 
Lemma 1 with Lemma 2 and the decidability of reachability for timed automata, 
the following theorem is immediate. 

Theorem 1. Reachability is decidable for {IZUVU Af)'' . 

Note that {IZUV U A/")'' is a proper superclass of the initialized rectangular 
hybrid automata, and that the possibility of further extension remains open. 
New building blocks may be added easily; they will slip right into place, as long 
as they are language equivalent to timed automata. Admittedly, this is a severe 
restriction. 



3 Irrational Timed Automata 

The semi-algebraic sets share many of the nice properties of the semi-linear 
sets [14]; in particular, they are closed under projection [13] and the boolean 
operations. So it is natural to ask if the results of the preceding section remain 
true in this more general context. 

Definition 11. We use Csa, TZ-sa, Us a and AIsa for the generalizations ofC, 
TZ, V and Af to semi- algebraic hybrid automata. 

As before, we can prove that every automaton Ag {IZsa U T>sa U A/sa)'^ is 
language equivalent to an automaton But note that Af is not necessar- 

ily a timed automaton; its constants are arbitrary algebraic numbers and may 
be irrational. So we are led to ask if reachability remains decidable for algebraic 
timed automata. Unfortunately, it does not. 
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Theorem 2. Reachability is undecidahle for 

Before preceding with a proof of this theorem, there is further motivation. 
Reachability is decidable for several classes of hybrid systems, for example [8] 
and [9]; we focus on two. We have already mentioned the initialized rectangular 
hybrid automata, and even offered a modest generalization. The second class con- 
tains the semi-algebraically defined hybrid automata with constant (set-valued) 
reset maps, which are proven to have computable finite bisimulations in [10]. 
To what extent can these classes be combined while preserving the decidability 
of reachability? Algebraic timed automata represent, in our opinion, a simple 
midpoint between these two classes, and in this light, the undecidability of the 
reachability problem presents an obstacle to a natural unification. 

3.1 Minsky Machines and Undecidability 

We prove our main theorem in more generality to illustrate that undecidabil- 
ity does not arise from some subtle property of the algebraics. Rather, it is a 
consequence of irrationality. This generality will also be useful in Sect. 4. 

Definition 12. Given 5 C M, the class Ts of irrational timed automata over S 
is the generalization of timed automata in which the guards and state invariants 
are allowed to have constants from Q U 5. 

In particular, = 7 a is the class of algebraic timed automata, where A 
is the set of all algebraic numbers, i.e. real roots of polynomial equations with 
rational coefficients. 



Theorems. Let r G (1>2) be irrational. Let S = {0,l,r, 3 — r}. Then the 
reachability problem for the class Ts is undecidable. 

Our proof of undecidability closely follows the technique in [7], where the 
undecidability of several slight generalizations of timed automata is proved. In 
particular, we proceed by reducing the halting problem for two-counter Minsky 
machines to the reachability problem for the class 7s . Before presenting this 
reduction, we give a definition of two-counter machines. It is well known that 
the halting problem for two-counter machines is undecidable [11]. 

Definition 13. A two-counter Minsky machine is finite state machine with two 
natural number counters c\ and C 2 . Each machine state has an associated com- 
mand which is executed when the machine is in that state. Possible commands 
are: 

• increment Ci and go to n 

• decrement Ci and go to n; if c, = 0 then it is unchanged 

• if Ci is zero go to n, otherwise go to m 

• halt 

where i G {1,2} and n, m are machine states. There is a distinguished start state 
and the machine begins its execution with both counters set to zero. 
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Xl = (cit) 
X2 = (C2t) 
c = 0 




Fig. 4. Halt 



Proof (Theorem 3). 

Let .4 be a two-counter machine. To simplify the encoding we can assume 
that it never decrements a counter containing zero. Of course, any two-counter 
machine can easily be modified to meet this restriction. 

Let (x) denote the non-integer part of a: € M. In particular, for any x, 0 < 
(x) < 1. We will encode the values of the counters ci and C 2 in continuous 
variables a;i and X 2 by representing the natural number n by the real number 
(nr). Because t is irrational, (nr) = (mr) if and only if n = m. 

We now construct a timed automaton A* G Tg. It will have three clocks 
components. We represent the continuous state of these components by xi, X 2 
and c. As indicated, xi an a ;2 store the counter values. 

In the construction of A*, each state of A is replaced with one of the four 
gadgets illustrated in Figs. 1-4, depending on its associated command. For ex- 
ample, a state with command “Increment C 2 ” would be replaced by the gadget 
in Fig. 1, but with the roles of x\ and X 2 reversed. In the figures, a state q is 
represented by a node labeled with the state invariant Tq. An edge e is repre- 
sented by an arrow from the node for s(e) to the node for h(e) labeled by both 
the guard for e and by the set of clocks reset to zero by the transition. 

We define the destination of edges leaving a gadget to correspond to the 
transitions in the two-state machine A. Finally, let S = (( 707 O)) where go is 
the discrete state in the gadget corresponding to the initial state of A. This 
completely specifies a timed automaton A * . 

The reader is encouraged to carefully examine Figs. 1-4 to understand why 
the gadgets that they depict have the asserted effects. It is worth noting that 
Fig. 2 is the same as Fig. 1 except that r is replaced by r' = 3 — t G (1, 2). It is 
also worth noting that the each gadget is defined to guarantee that c = 0 when 
the next gadget is entered. 

By construction, the two-counter machine A halts if and only if there is a 
reachable state of A* which corresponds to a halting state of A. As mentioned 
above, the halting problem for two-counter machines is undecidable. This proves 
that reachability is undecidable for the class Xs- n 

Theorem 2 is proved by letting r = -\/2 in Theorem 3 and noting that 

^ r~ A\ 

''{0,1,V2,3-V2} — ^SA- 
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3.2 Further Results 

Both TZsa and T>sa are extensions of Csa - Therefore, the undecidability of reach- 
ability for 7^1^ and follows from Theorem 2. On the other hand, re- 
quires a different proof. The following results are each proved by refining of the 
construction for Theorem 3. The gadgets become rather complicated to circum- 
vent the additional restrictions, but no other change is necessary. 

Theorem 4. 

1. Reachability is undecidable /or A/"!^ 

2. Let T > 0 be irrational. Then reachability is undecidable for (with as 

few as three clocks). 



Definition 14. Given 5i , 52 C K, let 7^,^ be the class of irrational timed 
automata with the first clock constrained by constants from Si and the remaining 
clocks constrained by constants from S 2 . 



Theorem 5. Let t > 0 be irrational. Reachability is undecidable for 
(with as few as four clocks). 

Before moving on, one simple decidability result should be mentioned. 

Theorem 6. Reachability for one-clock timed automata over M depends only on 
the order of the constants ( including zero ), and is decidable given that order. 

4 Parametric Timed Automata 

Without significant modification, the undecidability results of the previous sec- 
tion carry over to the context of parametric timed automata. These automata — 
introduced in “Parametric Real-time Reasoning” [2] — allow us to express a more 
sophisticated range of synthesis and verification questions, but their most ba- 
sic properties turn out to be undecidable [2]. After discussing the connection 
between parametric timed automata and timed automata with irrational con- 
straints, we state a new undecidability result and then examine the complexity 
of the one-clock case, for which reachability is decidable. 

Definition 15. 

(a) Parametric timed automata are a generalization of timed automata in which 
the guards and state invariants are allowed to have constants from Q U dr, 
where dr is a set of parameter variables. 

(b) Let A be a parametric timed automaton with parameters from dr and let 
A : dr — > Q. Then A\ is the timed automaton that results from using A to 
substitute for the parameters in A. 
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(c) If <7 is a state of A, then r^(^) is the subset of parameter space for which 
A has a run reaching q. In other words: 

rg(^) = {A:'I'^Q I q is a, reachable state of AIa}- 

Now consider what would happen were r a rational number in the proof of 
Theorem 3. In particular, let r = a/bG (1, 2), where a and b are relatively prime. 
As long as our virtual Minsky machine keeps its counter values below 6, nothing 
can go wrong. But a counter value of b is indistinguishable from zero; we have 
an overflow error. Such an error is easy to detect if we always test for zero after 
incrementing a counter. Thus, we can correctly simulate the Minsky machine as 
long as the counter values remain small and suspend the simulation when an 
overflow error is detected. 

At the risk of stating the obvious, note that if a Minsky machine halts then 
its counters remain bounded. Also note that the rational numbers in the interval 
(1, 2) have arbitrarily large denominators (in reduced form). Therefore, a Minsky 
machine halts if and only if that fact is detected by the simulation for some 
rational r G (1,2). With a few simple details swept under the rug, this is all it 
takes to translate the theorems of the last section into theorems about parametric 
timed automata. Under this translation. Theorem 4.2 becomes: 

Theorem 7. The emptiness ofTq(A) is undecidable for the class of parametric 
timed automata with three clocks and one parameter. 

This is not an essentially new result. That the emptiness of rq(A) for para- 
metric timed automata is undecidable was proved in [2] . The proof given there 
uses three clocks and six parameters but has the advantage of working for both 
the dense-time and discrete-time cases. The translation of Theorem 5 is more 
interesting. 

Theorem 8. The emptiness ofTq(A) is undecidable for the class of parametric 
timed automata with only one clock constrained by parameters. 

The corresponding problem is decidable for discrete-time [2], so we have 
exposed a divergence in the expressive power of dense-time and discrete-time 
parametric timed automata. 

Before leaving the subject of parametric timed automata, let us turn our 
attention to the one-clock case. Let A be a one-clock parametric timed automa- 
ton. As was noted before, the time-abstract runs of a one-clock timed automaton 
depend only on the order of the constants. So, to calculate rg(A), we simply de- 
termine the reachability of q in A\ for sample assignments A corresponding to 
every possible ordering of the constants (including zero) and parameters. Unfor- 
tunately, the number of such orderings^ grows exponentially in the number of 
parameters. 

^ Let Trjj, be the number of (non-strict) orderings that can be formed from n parame- 
ters with respect to m distinct constants. The following formulae generalizations of 
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We conclude with a number of observations about the complexity of the 
problem of determining emptiness for one-clock parametric timed automata. 
Both standard algorithmic complexity and parametric complexity [5] results are 
considered. 

Theorem 9. Consider the problem of determining the non-emptiness ofTq{A) 
for the class of one-clock parametric timed automata. 

1. The problem is NP-complete. 

2. For any fixed k bounding the number of parameters, the problem can be solved 
in polynomial time. 

3. Parameterized by the number of parameters, the problem is W[SAT]-hard. 
Note that it is strongly suspected that W[SAT] is a proper superset of the 
fixed parameter tractable (FPT) problems [5]. 

4 . Parameterized by the number of both constants and parameters, the problem 
is FPT. 



5 The Complexity of Questions About SLHA 



This last section deviates from the course of the paper thus far. The only con- 
nection it bears to the earlier sections is the reliance on Minsky machine simu- 
lations; they play a central role in proving the hardness directions of all of the 
completeness results that follow. 



Definition 16. A maximal run is any run that can not be extended. It either 
has an infinite number of transitions, ends with an infinite time interval spent 
in the same discrete state, or reaches a state from which it can neither flow nor 
jump. A maximal run is said to be jump-infinite if it makes an infinite number of 
discrete transitions; otherwise jump-finite. It is time-infinite if its time intervals 
sum to infinity; otherwise time-finite. A maximal run is blocking if it is both 

those in [6], where the sequence {ttq }«=n of preferential arrangements is studied. 



7T 



n 

m 








n\ k 



TTm = ^ -I- ^ 

k=0 

Finally, writing / g to denote lim f /g = 1, 

n — >oo 

(n -I- m)! 



(Vm) 7 t” 



‘ 2m! ln"+’"+i 2 
(Vn) Trjji ~rn(2m)". 
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Table 1. The complexity of detecting various types of runs 



type 
of run 


Semi-linear Deterministic 

Hybrid Compact Deterministic and Non-blocking 

Automata SLHA SLHA Non-blocking SLHA 

(SLHA) SLHA 


Zeno 

time- finite 




S 2 -complete 


Sj-complete 


non-Zeno 

time-infinite, 
jump- infinite 

time-infinite 


Si-complete 


H 2 -complete 


jump- infinite 








infinite 


Hi-complete 


always 


arbitrarily 

long 






arbitrarily 
long blocking 


H 2 -complete 




never 


blocking 






time-infinite, 

jump-finite 

jump-finite 


El-complete 





Table 2. Supplement to Table 1 



Semi-linear ■,,, , , 

Hybrid Determinfstic Deterministic 

Automata ottua oLHA 

(SLHA) 


Zeno run 


Sj-complete 


S 2 -complete 


time-infinite 
run for every 
initial state 


n2-complete 


Hi-complete 


H 2 -complete 
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time-finite and jump-finite; otherwise we call it infinite. Finally, a maximal run 
is Zeno if it is time-finite but jump-infinite. 

A hybrid automaton is said to have arbitrarily long runs if for each n G N 
there is either a run that makes at least n transitions or a run with a duration 
of at least n. 

Definition 17. An SLHA is compact if all of its defining regions are compact. 
An SLHA with at most one initial state and at most one possible evolution from 
each state is called deterministic, and an SLHA with at least one initial state 
and at least one evolution from each state is called non-blocking. 

Table 1 gives, for different classes of SLHA, the complexity of determining 
whether certain types of runs exist. It is only a sampling of complexity results. 
Further questions might prove interesting; for example, “Is there an time-infinite 
run for every initial state?” 

Also, our definition of determinism is very restrictive. A more reasonable 
property is that there is at most one evolution from each state, with no restriction 
made on the initial set. We call this property weak determinism. Table 2 shows 
that questions may be much harder for weakly deterministic SLHA than for 
deterministic SLHA. The complexity of many questions matches that of general 
SLHA, but this is not always the case. 

As a closing note, the Zeno phenomenon is exploited in [3] and [4] to show 
that the reachability problem for dynamical systems with piecewise constant 
derivatives is arithmetic and hyper-arithmetic, respectively. 
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Abstract. Reachability analysis is frequently used to study the safety of 
control systems. We present an implementation of an exact reachability 
operator for nonlinear hybrid systems. After a brief review of a previ- 
ously presented algorithm for determining reachable sets and synthesiz- 
ing control laws — upon whose theory the new implementation rests — an 
equivalent formulation is developed of the key equations governing the 
continuous state reachability. The new formulation is implemented using 
level set methods, and its effectiveness is shown by the numerical solution 
of three examples. 



1 Introduction 

The reachability operator, a function or algorithm that can determine the evo- 
lution of sets of trajectories, is key in the synthesis and verification of controllers 
for continuous, discrete or hybrid systems. Regardless of whether reachability 
appears implicitly, such as in the generation of invariant sets, or explicitly, no 
technique for determining safe control systems can avoid its use. It is natural 
that methods for its accurate, automatic computation are attracting consider- 
able attention. 

Reachability analysis of hybrid systems has been investigated by both the 
computer science and control communities. Methods have been developed by 
computer scientists for computing reachable sets for timed automata [1] and 
linear hybrid automata [2], for which computation is based on the propaga- 
tion of polygonal sets under constant rate dynamics. Tools have been developed 
to perform such calculations automatically [3,4], and to synthesize controllers 
in such a framework [5,6]. Control theorists have extended reachability tools 
from continuous state and time dynamical systems theory to incorporate dis- 
crete switches [7,8,9,10,11]. However, the efficient computation of reachable sets 
for hybrid systems with nonlinear dynamics remains a difficult problem to solve. 

* Research supported by DARPA under the Software Enabled Control Program 
(AFRL contract F33615-99-C-3014), and by a Frederick E. Terman Facnlty Award. 



N. Lynch and B. Krogh (Eds.): HSCC 2000, LNCS 1790, pp. 310—323, 2000. 
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Numerical techniques which over-approximate the nonlinear dynamics with lin- 
ear dynamics [12], or which over- approximate the reachable sets [13,14,15,16], 
have recently been developed. 

In this paper, we present an implementation of an exact reachability operator 
for nonlinear hybrid systems. An algorithm which synthesizes control laws for 
such systems based on the Hamilton- Jacobi equation [9,10,11] is reviewed, and 
then a new Hamilton- Jacobi formulation with superior numerical properties is 
developed and proved to be equivalent. While level set techniques were previously 
investigated for the solution of such equations in [17], we have added several 
improvements to the basic level set algorithm. Examples from [11] demonstrate 
the results of applying the new algorithm to the new equations — examples which 
have never previously been solved computationally. 



2 Deriving Reachable Sets in Hybrid Automata 

In [11], an algorithm is presented which characterizes the reachable set of a non- 
linear hybrid automaton (with desired safety properties) as that whose boundary 
is the zero level set of a particular Hamilton- Jacobi equation. The algorithm also 
computes the continuous and discrete control laws to maximize the safe oper- 
ating region. In this section, we briefly review this hybrid system model and 
reachability algorithm, and then present a second characterization using a simi- 
lar Hamilton- Jacobi algorithm with better numerical properties. 



2.1 Hybrid Automata and Hamilton-Jacobi Equations 

A hybrid automaton is defined as 

H={{Qx X), (UxD), (A„ X Ed), /, <5, Inv, ^2) (1) 

where Q is a finite set of discrete states, X = K”, U C is the set of continuous 
control inputs, D C M”'' is the set of continuous disturbances, E = E^ x Ed is 
a finite set of actions, where denotes the set of discrete control inputs, and 
Ed the set of discrete disturbance inputs, f: QxXxUxD^ M” defines the 
flow of continuous trajectories, 6 : Q x X x x Ed ^ 2*3^^ is the discrete 
transition function, Inv C Q x X is the invariant associated to each discrete 
state, and Q is an acceptance condition — here 12 = (nE), meaning that the 
state of the system must remain within a set F C Q x X . We denote U as the 
set of piecewise continuous functions from M to U, and V the set of piecewise 
continuous functions from M to U. 

Three operators are defined: 

Preu{K) = {{q,x)eQxX\3au € Eu Wad&Ed 5{q,x,au,crd) C K} n K 
Pred{K) = {{q, x) e Q X X\\/au G Eu 3ad G Ed 6{q,x,au,(Jd) H U 

Reach{G,E) = {{q,x) G QxX \ \/u GU 3d GV and t>0 such that 

(q{t), x{t)) G G and {q{s),x{s)) G Inv \ if for s G [0, t]} 
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where K C Q x X; G,E C X; and (g(s),a;(s)) is the continuous state trajectory 
of i = f{q{s),x{s),u{s),d{s)) starting at (q,x). The set Reach{G,E) describes 
those states from which, for all u(-) GU, there exists a, d{-) G T>, such that the 
state trajectory ((/(s), a;(s)) can be driven to a “bad” set G while avoiding an 
“escape” set E. With these definitions in place, the algorithm for reachability 
analysis for hybrid systems proceeds as follows [10,11]: 

Let = F, W-i = 0,t = 0. 

While W* + do 

^/i-l = PL* \ Reach(Pred(W^), Pre„(W*))) 
i = i — 1 

end 

If the algorithm terminates after a finite number of steps, then the fixed point 
W* is the largest set of states for which the control (m(-), cr„[-]) can guarantee 
that the state of the hybrid system remains inside E despite the action of the 
disturbance (d(-), <Td[-]). In order to implement this algorithm, Pre„, Prcd, and 
Reach need to be computed. The calculation of Pre„ and Prcd requires inversion 
of the transition relation 5 subject to the quantifiers 3 and V. The computation 
of Reach requires an algorithm for determining the set of initial conditions from 
which trajectories can reach one set, avoiding a second set along the way. Our 
focus in this paper is on numeric computation of the latter operator. 

Let Iq '■ X ^ and Ie ■ X ^ R he differentiable functions such that 

G = {x G X\la{x) < 0} and E = {x G X\Ie{x) < 0}. Consider the following 
system of interconnected Hamilton- Jacobi equations [11,17]: 

dJ^{x,t) jH*g{x,^^), for {xGX\J*a{x,t)>0}, 

dt I min{0, H^{x, for {x G X \ J^{x, t) < 0} 

dJ*E(x,t) ^ I H%{x, ^ for {x G X I J%{x,t) > 0}, 

dt j min{0, H%{x, for {x G X \ J%{x, t) < 0} 

where Jq{x,u{-), d{-),0) = Ig{x) and J^{x,u{-),d{-),0) = Ie{x), and 

H* = l or ^ ^ (4) 

^ ' dx \max„g£/mindgD ^/(a;,M,d), otherwise 

= gj, for {a; G X I J^(a;,t) < 0} 

^ ’dx \min„g£/maxdgzj ^/(x,M,d), otherwise 

Theorem 1 (Characterization of Reach- Avoid [11]) Assume that jQ{x,t) 
(Je(xA) respectively) satisfies the Hamilton- Jacobi equation (2) ((3) respec- 
tively), and that it converges uniformly in x as t ^ — oo to a function Jq{x) 
(Je(^) respectively). Then, 

Reach{G, E) = {x G X \ J^(x) < 0} 



(6) 
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Proof. Please see [11]. □ 

By our convention, we assume that the unsafe sets, defined as G° and its back- 
wards reachable set under (2)-(5), are open; and safe sets, defined as E and its 
backwards reachable set, are closed. 



2.2 An Equivalent Hamilton-Jacobi Formulation 

Although the Reach operator can be computed by solving the equations (2)-(5), 
in practice the discontinuous right hand sides of the equations introduce serious 
numerical instabilities into the computation. Consider instead the standard form 
of the Hamilton-Jacobi equation: 



dJa{x,f) 

dt 

dJE{x,t) 

dt 



Hg{x, 



He{x, 



dJG{x,t) 
dx ’ 
dJE(x,t) 
dx ’ 



■ rf A 

= max mm -- — fix, u, a), 
ueu den dx ^ ’ 

9Je j ., n 

= mm max — — f[x,u,d), 
ueu deD dx ^ ' 



( 7 ) 

(8) 



with the same initial conditions as those used for Jq and Jg{x,Q) = la{x) 



and Je{x,0) = Ie{x). Now let: 

jQ^"^{x,t)= min Jg{x,t), (9) 

re[i,0] 

JE^'^{x,t)= min Je(x,t), (10) 

re[i,0] 

JG{x,t) > (11) 

JE{x,t)>-J^^{x,t). (12) 



Constraints (9) and (10) replace the “min” on the right hand side of equations 
(2) and (3), thus ensuring that sublevel sets of J)j‘”(a;, t) and Jff™{x,t) do not 
shrink as time flows backwards; constraints (11) and (12) replace the “freezing” 
of the Hamiltonian on the right hand sides of equations (4) and (5) and ensure 
that the interiors of the two sets do not overlap, since for a given x G X, if 
Jff^^{x,t) < 0, then (11) will force JG{x,f) > 0; conversely, if jQ™{x,f) < 0 
then Je{x, t) > 0. 



Lemma 1 (Equivalence of Solutions) The solution jQ{x,t) to (2)-(5), and 
the solution jQ^'°^{x,t) to (7)-(12), are equivalent in that, for any x G X, they 
satisfy one of 

Jg(x, t) <0 if and only if J)j‘”(a:, t) < 0 (13) 

Jg(x, t) < 0 if and only if J)?“(a:, t) < 0 (14) 

for all t < 0. 



Proof. We choose a particular x G X and assume that the computation starts 
at final time t = 0 and works backwards into negative time. Also, assume that 
the interiors of the initial sets do not intersect: G° C = 0. 




314 



I. Mitchell and C.J. Tomlin 



Case 1 (a; is in G at t = 0). Thus Icix) < 0, which implies from (2) Vt < 0 
that jQ^Xjt) < 0 and from (9) that jQ™{x,t) < 0. Thus, for such x, (13) holds. 

Case 2 (a; is in if at t = 0). Thus Ie{x) < 0, meaning that J|;(a:, 0) < 0 
and Je{x,0) < 0, and in addition, due to our assumption that the interiors 
of the initial sets are disjoint, Jq{x,0) > 0 and Jg{x,0) > 0. By (3), Vt < 0 
J^{x,t) < 0, and so by (4) jQ{x,t) = Jq{x,0) > 0. In our new formulation, 
Je{x,0) < 0 implies J)^“(a;,t) < 0 Vt < 0; by (11) Jaixjt) > 0, which in turn 
implies jQ^^{x,t) > 0,Vt < 0. Thus, Vt < 0, JqIx^I) > 0 and J)j“(a:,t) > 0. By 
the contrapositive, for such x, (14) is true. 

Case 3 {x is outside both G and E at t = 0). Thus, la{x) > 0 and 
Ie{x) > 0. Now, for all t < 0, x will remain outside both the reach and avoid 
sets as long as the following constraints are satisfied: 

J^{x,t)>0 Jl,{x,t)>0 . . 

> 0 JE^{x,t) > 0 ^ ^ 

For an x under these conditions, (13) is trivially true. Furthermore, while this 
situation holds, the constrained PDFs (2)-(5) are equivalent to the PDFs and 
constraints (7)-(12), and so jQ{x,t) = Ja{x,t) and J^{x,t) = JE{x,t). Now 
consider what will happen if the boundary of one or both of the reach or avoid 
sets reaches x. Choose t < 0 to be the first time t that either boundary reaches 
x. 

If Jq{x, t) = Jg(x, t) = 0, then (2) guarantees Jg(x, t) < 0 for t < r and (9) 
guarantees jQ™{x,t) < 0 for t < r. Consequently, for such x, (13) holds Vt < 0. 

By choice of t, we know that if J^{x,t) = Je{x,t) = 0, then Jq{x,t) > 0 
and Jq'’^{x,t) > 0. By (3), Vt < r, J^{x,t) < 0, which implies by (4) that 
Jq^XjI) > 0. Since Je(x,t) = 0 implies Wt < t that Jg™(a;,r) < 0, (11) 
requires yt < t that JG{x,t) > 0, and so jQ"^{x,t) > 0. Therefore, for such 
x, (13) holds for T < t < 0 and (14) holds for t <t. □ 

We wish to use Lemma 1 and Theorem 1 to claim 

Reach{G,E) = {x & X \ Jg‘“(a;,t) < 0}. 

However, the two cases (13) and (14) allowed by Lemma 1 must be reconciled 
before such a claim is true. We do so by making the assumption that the sets 
defined by (13) are the closures of the sets defined by (14)^. 

Given this assumption, the formulation (7)-(12) provides a characteriza- 
tion of the reach-avoid operator which is numerically more stable than (2)-(5). 
While the new formulation does smooth out the solution of the Hamilton- Jacobi 
equations, it is worth noting that discontinuities in u, d, or / will still lead to 
non-smooth solutions of (7)-(12), and that even if these system parameters are 
all smooth, it is possible for discontinuous “shocks” to develop as the solution 
evolves. 

^ This assumption will hold true as long as the functions Jq and Jq'” do not develop 
plateaus. It turns out to be prudent to avoid plateaus for numerical reasons as well, 
and we describe a method to avoid their formation in the next section. 
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3 Computing Reachable Sets 

The continuous Hamilton- Jacobi partial differential equation appears frequently 
in applied mathematics, and so numerical methods for its solution have been well 
studied [18] . In particular, a set of algorithms called level set methods [19,20] have 
been developed to study the propagation of moving interfaces and boundaries 
using these equations. 

A numerical algorithm to solve the Hamilton- Jacobi equations (7)-(12) was 
developed in [17]; however, the emergence of numerical instabilities meant that 
the reach set could be computed for only a few dozen timesteps, and even over 
that short period, sharp edges tended to become rounded by diffusion. Armed 
with the better behaved (7)-(12) and a new level set implementation, we are 
able to tackle more complex examples below, tracking the reach set over any 
finite time interval without significant loss to diffusion. 

3.1 Level Set Method Design 

The basic method for solving (7) and (8) is the same as that described in [17]: a 
first-order, upwinding, finite difference scheme that produces an approximation 
of the viscosity solution to the Hamilton- Jacobi equation [20,21,22]. We outline 
several details of our implementation. 

Initial conditions: A characteristic of level set methods is that the “level 
set function” (we use J in the following to represent Jq in (7) or Je in (8)) is 
defined as the distance to the boundary being tracked, where distance is negative 
on the inside of the boundary. Such a definition is compatible with the analysis 
in the previous section, and so we adopt it for our level set functions. 

Boundary conditions: The spatial derivatives in the Hamilton- Jacobi equa- 
tion are approximated at a grid point by taking differences between the function 
values at neighboring grid points. For points at the edge of the finite grid, this 
procedure breaks down. Typical level set methods use Neumann boundary con- 
ditions ( = 0) where n is an outward pointing normal) to determine the 
value of grid points on the boundary. This procedure tends to introduce plateaus 
to the level set function J close to the boundary, so that it no longer properly 
measures the distance to the boundary. 

Enforcing the constraints: To enforce the constraints (11) and (12), a 
“max” operator is applied: at each timestep t, for all x, 

Ja{x,t) = max(JG(a;,t), -J™(a:,t)) 

and similarly for JE{x,t). This procedure, called masking Jq with Je™i is used 
in level set methods to ensure that the moving boundary represented by Jq 
does not enter the forbidden region defined by T™" (since J^^’^{x,t) < 0 
Jcix.t) > 0 ). 

An additional complication arises from the discrete timesteps taken by the 
numeric solver: it is possible for the constraints (11) and (12) to become vi- 
olated since the various J functions are changing over time and the masking 
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procedure is only applied at the end of each timestep. A conservative solution is 
to compute (7)-(12) in the order: 

compute Jc(x,t — At) from Hamilton- Jacobi equation, 
compute JE(x,t — At) from Hamilton- Jacobi equation, 

Ja{x,t — At) = max(JG(a;,t — At), — 
jQ™{x,t — At) = min(JG(a;,t — At), 

Je{x, t — At) = max( j£;(x, t — At), — t — At)) 

JE™{x,t — At) = min(j£;(a;,t — At), J^'’^{x,t)) 

Masking Jq with Je^’^ from the previous timestep, but masking Je with Jq'” 
from the current timestep ensures that if the reach and avoid sets grow together 
and overlap, the reach (unsafe) set is over-approximated, and the avoid (safe) 
set is under-approximated. 

Reinitialization: Level set methods attempt to maintain the level set func- 
tion as a distance measure to the boundary as it evolves. Numeric solutions tend 
to distort the distance function considerably: the level set function becomes 
distorted by limited precision computations, discretization and the Neumann 
boundary conditions. Because the zero level set is the only information of im- 
portance to us, a procedure which resets the level set function so that it correctly 
measures the distance to the current zero level set — without changing the shape 
of that level set — would smooth out numerical errors in the level set function 
and yet leave its important data unharmed. This process, called reinitialization, 
is accomplished in the examples below by running a few discrete timesteps of a 
solver for the partial differential equation 

= sign{JG{x,t)){l - |grad(Jc(a:,t))|) 

(and similarly for Je)- This process restores the property | grad(JG(a;, t))| « 
1 near the zero level set, so that Jq is smoothed to approximate a distance 
measure. 

3.2 A Single State, Straight Flight Example 

Consider an example representing two aircraft flying at a fixed altitude and con- 
stant heading. Each aircraft is allowed to choose its own speed from a given range 
of values; we control one aircraft and the other is considered the disturbance. 
Using relative coordinates, in which the controlled aircraft is at the origin with 
a heading angle of zero, the dynamics of the system are described by 

Xr = —U + dcosipr, yr = dsm^r, V’r = 0, (16) 

where Xr and yr are the relative spatial coordinates, and 'tpr is the relative 
heading. The controller fails if the disturbance aircraft manages to enter a circle 
of radius five units centered at the controlled aircraft at the origin, so la(x) = 
X^+Vr — 5^. 
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If the control (speed of the controlled aircraft) is restricted to u G U = 
[u, u] C M"*" and the disturbance (speed of the disturbance aircraft) is restricted 
to d G D = [d,d] C M“*", then it was shown in [11,23] that the optimal control 
and worst disturbance are 



Because there is only a single discrete state, the controlled aircraft has no discrete 
action to force an unsafe continuous state to become safe, and so the avoid set 
is empty. Given the definition of the unsafe set G = {x G X\la{x) < 0}, the set 
of unsafe states Reach{G,%) is shown shaded in Figure 1. The parameters for 
the example were chosen to be the normalized values: 

V’r = ^, U =[u,u] = [2,A], D =[d,d] = [l,5]. 

The dashed circle shows the initial unsafe set G, and the grey arrows show the 
flow field (16) induced by the optimal control choices (17). Notice that the level 
set algorithm resolves the sharp corners of Reach{G, 0) at the points where u* 
or d* switch. 

This example and those below were coded in Matlab 5.3 on an unloaded Sun 
UltraSparc 10 (a 300 MHz UltraSparc processor with 512 KB cache and 128 MB 
main memory). Figure 1 was produced from a run with grid spacing Ax = 0.1 
(requiring about 63000 grid points). The 360 timesteps took just under four 
minutes to complete. 

3.3 A Three State Example 

This example again features the collision avoidance maneuvers of two aircraft at 
fixed altitude; however, the control is now allowed to initiate a discrete change 
of state for the system. As shown in Figure 2, the aircraft begin in straight flight 
at a fixed relative heading (mode 1). At some time, the control may switch both 
aircraft into mode 2; at which point each makes an instantaneous heading change 
of 90°, and begins a circular flight path. After completing a semicircular arc in 
7T time units, both aircraft switch to mode 3, make another instantaneous 90° 
turn, and resume their original headings from mode 1. 

The dynamics for the system are shown in Figure 3. In this example, the 
controller has only a single action: the switch from mode 1 to mode 2. The speed 
of both aircraft is constant, and the only disturbance action is the uncontrolled 
switch from mode 2 to mode 3, which occurs a fixed time after mode 2 is entered; 
the variable 2 in mode 2 is simply a clock to enforce this switch. The parameters 
used in the run below are 

Stt 

iPr = — = 120°, u* = 3, d* = 4. 

o 

More details on this example can be found in [9,11]. 



U, if Xr > 0, 
U, if Xr < 0, 



d, if {Xr cos 1pr + Ur sin Ipr) > 0, 
d, if {Xr cos 1pr + Ur sin tpr) < 0. 
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Fig. 1. Shaded Region represents Reach{G, 0) for the Straight Flight Single 
State Example 



Mode 1 




Mode 2 Mode 3 




Fig. 2. Aircraft Behavior in the Three Modes 



Running the reachability analysis algorithm to compute W* requires com- 
puting the Prcd and Pre„ operators for each mode. Let Rf be the set of unsafe 
states computed for mode i in iteration k; in other words, the projection of 
Reach{Pred{W^~^^), Preu{W’^~^^)) onto the continuous state space of mode i 
for iteration k < 0 (let R^ = G to handle the k = 0 case). Then the set of safe 
states at iteration k < 0 can be written as RiY- Define the 

collision set as before: G = {x G X\lc{x) < 0}, where Ig{x) = x'j + — 5^. We 

can then deduce the precursor operators. 

— For mode 3, there are no discrete actions. This mode may be inhabited 
for any length of time. The projections of the precursor operators onto the 
continuous state space of mode 3 are: 

PreuiWY = 0 , PrediwY = R^ 

— For mode 2, an uncontrolled discrete action switches the system to mode 
3, and there are no controlled discrete actions. This mode is inhabited for 
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Xr = -U + d COS¥r 
yr= d sin 

¥r=0 

z=0 



Xr= -U+d COS¥r + 
yr= d sin^r- Xr 

'z =1 



Xr = -U +dcOS¥r 
yr= d sin ¥r 
¥,= 0 
z =0 



Fig. 3. System Dynamics for the Three Mode Example 



exactly tt time units. The projections of the precursor operators onto the 
continuous state space of mode 2 are: 

PreuiW^) = 0 , PrediW^) = {R 3 rotated f ) U R^. 

— For mode 1, a controlled discrete action switches the system to mode 2, 
and there are no uncontrolled discrete actions. This mode may be inhabited 
for any length of time. The projections of the precursor operators onto the 
continuous state space of mode 1 are: 

PreuiW'^) = (i ?2 rotated f Pred{W'^) = R^ 

Figure 4 shows the results of the reach-avoid computation at each iteration 
for each mode; unsafe states (complement of W^) are shaded. The set appears 
in column i and row k. A fixed point W* of safe states is computed after three 
iterations, and the corresponding bad states of the fixed point {W*Y are shaded 
in the final row of plots. 

The unsafe region for mode 1 is the most interesting — as long as the distur- 
bance aircraft is not in this region, the control may initiate the switch to mode 
2 and have confidence that the remainder of the maneuver will be carried out 
safely. The width of the unbounded portion of the unsafe set is controlled by the 
radius of the turn in mode 2, and can be removed entirely by making the radius 
large enough. 

The four iterations of this simulation, with a grid spacing of Ax = 0.1 (or 
about 90000 grid points) each required about 1400 timesteps; for stability rea- 
sons, mode 2 was slightly more than half of the work. Wall clock time was about 
75 minutes. 



3.4 A Three Dimensional Example 

To show that this technique extends easily to higher dimensions, we look at a final 
aircraft collision avoidance scenario. The model is very similar to that examined 
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Mode 1 Mode 2 Mode 3 

Fig. 4. Unsafe Sets for Three Mode Example 



in the first example, except that this time we allow the relative heading of the 
aircraft to change. Relative angle ipr € [0, 27t) is thus our third dimension. 

We fix the airspeed of the control aircraft at v\ and that of the disturbance 
aircraft at V 2 - The control and disturbance inputs are now the angular velocity 
of the aircraft: u G U = aJT] C M and d G D = [^,W 2 ] C M. The model is 

Xr = —Vi + V2 cos 1pr + j Vr = V2 sin 1pr — UXr, Ipr = d — U, 

For the case where 



^ = ^=—1, = W2 = +1, 

it was shown in [11, pp. 60-62] that the optimal control and disturbance are 
given by 



u* = sign 



( dJa dJa 

dXr dyr 



dJg \ 

dtpr ) 
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Fig. 5. Unsafe Region for Three Dimensional Example 



Because there is only a single discrete state and no discrete actions, the avoid 
set is empty; the unsafe set is the cylinder G = {x G X\lc{x) < 0} where 
Ig{x) = x"l +y"^ — 5^. A view of Reach{G,%) for airspeed = W 2 = 5 is shown 
in Figure 5. 

Extending the level set code to three dimensions was painless — a new index 
for all matrices and a set of boundary conditions (periodic in 'tpr) had to be 
added. Visualization of the zero sublevel set becomes considerably trickier, but 
it can be done with Matlab’s new isosurface tools. With a grid spacing of Ax = 
0.2 (approximately 400000 grid points), the 400 timesteps required to generate 
Figure 5 took about 80 minutes to complete. 

4 Research Directions 

We have presented a numerical algorithm for computing reachable sets of hybrid 
automata. The algorithm handles nonlinear dynamics with discontinuities, as 
illustrated by example calculations of both continuous and multi-mode aircraft 
conflict resolution maneuvers. We are currently investigating further in several 
directions. 

For the examples above, the discrete predecessor maps Pre were determined 
by hand and hard-coded into the scripts which computed the continuous reach- 
avoid operator. It is necessary to automatically compute those maps; this will 
require elimination of existential and universal quantifiers over the set of discrete 
actions. 
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As with all finite difference methods, this implementation finds an approxi- 
mation to the actual solution of the Hamilton- Jacobi equation. In fact, the final 
example provides proof of the dangers of such approximations: the helical bulge 
of the unsafe set shown in Figure 5 is computed to protrude farther out if grid 
spacing is reduced. Methods to quantify the error between exact and approxi- 
mate reachable sets have not been developed, yet are crucial for proving safety 
properties. In the reach-avoid calculation, we could use information about error 
to provide an over-approximation of the unsafe set and an under-approximation 
of the safe set. 

In [9,10,11], control laws are synthesized assuming that the reach and avoid 
sets are computed exactly. The implications of set approximation on this process 
must be evaluated. 

As can be seen from the final example, these techniques extend easily to 
higher dimensions — beyond three dimensions visualization becomes impossible, 
but the basic level set algorithm remains the same. Of major concern, though, 
is the exponential growth in the number of grid points as dimension increases. 
Because the timestep depends on the grid size, using rectilinear gridding with 
a grid spacing of /i in d dimensions requires work. However, we are 

currently investigating techniques which will lead to considerable time savings: 
using compiled code instead of Matlab, computing only on grid points near the 
zero level set (effectively reducing the dimension of the problem by one) , taking 
advantage of the abundant opportunities for parallelism in the algorithm, and 
projecting higher dimensional sets onto lower dimensional subspaces. 
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Abstract. In many cases, complex system behaviors are naturally mod- 
eled as nonlinear differential equations. However, these equations are 
often hard to analyze because of “stiffness” in their numerical behavior 
and the difficulty in generating and interpreting higher order phenomena. 
Engineers often reduce model complexity by transforming the nonlinear 
systems to piecewise linear models about operating points. Each oper- 
ating point corresponds to a mode of operation, and a discrete event 
switching structure is added to implement the mode transitions during 
behavior generation. This paper presents a methodology for systemat- 
ically deriving mixed continuous and discrete, i.e., hybrid models from 
a nonlinear ODE system model. A complete switching specification and 
state vector update function is derived by combining piecewise lineariza- 
tion with singular perturbation approaches and transient analysis. The 
model derivation procedure is then cast into the phase space transition 
ontology that we developed in earlier work. This provides a systematic 
mechanism for characterizing discrete transition models that result from 
model simplification techniques. Overall, this is a hrst step towards au- 
tomated model reduction and simplification of complex high order non- 
linear systems. 



1 Introduction 

Systems and control engineers often apply simplification techniques when model- 
ing and analyzing complex physical systems that include components like valves, 
pumps, and diodes, and phenomena such as friction effects [3]. To avoid com- 
plex nonlinearities and stiffness caused by steep slopes in the behavior, these 
components are modeled to exhibit switching behavior. This results in the over- 
all system model generating piecewise continuous behaviors and discrete tran- 
sitions, i.e., hybrid behaviors. Hybrid automata [1] have been employed as a 
computational mechanism for implementing these models, with a discrete con- 
trol structure defining the switching between modes or states of the automata. 
Each mode has an associated set of ordinary differential equations (ODEs) that 
governs continuous behavior evolution in that mode. Events associated with the 
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mode switching generate actions that may produce discontinuous changes in 
state variables. 

Consider the hydraulic actuator illustrated in Fig. 1. The valve at the top 
of the cylinder controls oil flow into and out of the cylinder, and the flow rate 
is a function of the control pressure Pm ■ The flow of oil determines the position 
of the piston in the cylinder, and this in turn determines the position of the 
load, e.g., the elevator control surface of an airplane. To prevent damage to the 
actuator system, a relief valve on the left side of the cylinder opens when the 
pressure in the cylinder exceeds a certain value. 




Fig. 1. Model parameters of a hydraulic actuator. 



If the valve behaviors are approximated and simplified to be discrete, the 
actuator can be modeled as a hybrid automata with four states: ooo, both valves 
closed, aoi, relief valve open and control valve closed, oio, control valve open 
and relief valve closed, and an, both valves open. The dynamic behavior in each 
of these modes can be derived from the actuator parameters, that include i?i, 
the resistance of the open control valve, i?2, the internal dissipation parameter 
for the oil, i?3, the resistance of the open relief valve, C, the oil elasticity, Ji, the 
piston inertia, and I2, the relief valve fluid inertia. 

System modelers often employ simplification techniques that involve drop- 
ping very small and very large parameters that do not play a significant role in 
gross system behavior. Applying this approach to the actuator system, parame- 
ters associated with the oil, R2 and C, may be removed to reduce the order of 
the system model. For the simplified model, the dynamic behavior models for 
the different modes are given in Table 1, where fi is the piston velocity and /2 
is the fluid flow rate through the relief valve. The control valve and the relief 
valve are the two components in the actuator that are modeled to have discrete 
transitions from open to closed, and vice versa. An external control variable, 
determines the opening and closing of the control valve (e.g., the valve is closed 
when Uy < 0). The relief valve opens when p > pth- 

For mode aoo, there is no oil flow into the cylinder, therefore, the entry ac- 
tion, i.e., the initial conditions that have to be satisfied on entry into this mode, 
includes the constraint, fi = 0. The entry action for mode aoi is more compli- 
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Table 1. Mode specification table. 



mode 


X = f{x,u) 


entry action 


ooo 


II II 
0 0 


0 0 
II II 


OOl 




fi = - hf2) 

/2 = 7rT7j(^i/i - ^ 2 / 2 ) 


OlO 


. 1 - 

II 


/2 =0 


ail 


fl = i^{Pin-Rl{fl+f2)) 
h = T;(Pin - Rl{h + / 2 ) - A 3 / 2 ) 





cated. In this mode, /i and /2 are algebraically related (/i = — / 2 ). The initial 
values for fi and /2 have to be initialized using this constraint, but one equa- 
tion is not sufficient to solve for their values. Additional constraints presented 
in Section 5 are used to define the entry action listed in Table 1. 

In the past, engineers have used ad hoc approaches to handle transitions be- 
tween piecewise models, however, even for the simple example above this may 
lead to incorrect model definitions. In Section 5 systematic analysis shows that 
the entry actions as specified in Table 1 are incomplete, and demonstrates how 
the correct state mapping as derived by a structured approach is much more com- 
plex. This shows that deriving the correct event structures and corresponding 
actions at mode transitions is more involved for systems with complex interac- 
tions among their subsystems. 

This paper develops a structured approach to analyzing complex nonlinear 
models, applying systematic abstraction and simplification mechanisms to cre- 
ate simpler multiple piecewise continuous models. The price we pay in achieving 
this reduction is the introduction of complex discrete components in the hy- 
brid model of the system. The two main steps in this procedure are illustrated 
in Fig. 2. We start with the complex continuous nonlinear model of the sys- 
tem. Step 1 applies simplification techniques to convert the nonlinear models to 
simpler piecewise continuous (possibly linear) behavior models. The result is a 
hybrid model whose state variable values are continuous, but the time deriva- 
tives may be discontinuous. This is equivalent to a hybrid model with sets 
of differential equations defining the behaviors in individual modes, and a func- 
tion, 7 , that defines transitions between the modes. Step 2 applies techniques 
like singular perturbation [3] and eigenvalue analysis [ 8 ] that remove large and 
small parameters from the models, and thus eliminate steep transitions in the 
behaviors within modes. The resultant models combine three components: (i) a 
reduced order ODE model, /, (ii) the discrete event mode transition function, 7 , 
and (iii) the state transition function, g, that captures the discontinuous state 
variable value changes between modes. 

The derivation process for g can be described by two basic actions in hybrid 
models of physical systems: 
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Fig. 2. Abstraction levels. 



1. a manifold projection that results from the generated algebraic constraints, 
and 

2. an aborted projection because detailed continuous projection behavior causes 
further discrete changes. 

We use this framework to derive a computational model of the resulting hybrid 
system as a hybrid automata extended with branch points (junctions) to model 
the immediate consecutive discrete events and actions. A phase space analysis 
illustrates these concepts, and allows us to relate the results back to an ontology 
of phase space transition behavior presented in previous work [6] . 



2 The Approach 

Consider a nonlinear system with state equations of the form 

X = A{x)x + B{x)u. (1) 

System designers and analysts often simplify the above model by identifying 
operating regions of interest within the behavior space, called modes. Such modes 
may be the result of design decisions, e.g., the take-off, cruise, and landing modes 
of aircraft fly-by-wire systems, or determined from component models that make 
up the system, e.g., by taking into account the open and closed states of the 
valves in the actuator system. Modes can also be identified by the discrete control 
actions of supervisory controllers. Along with mode identification, transitions 
between modes, a, are also defined (cf. Table 1). Most often, the purpose for 
breaking up complex behaviors into modes of operating regions is so that the 
system model can be linearized within each mode, i.e., 

foii ■ X = Aa-X + Ba^U. (2) 

The result is a set of piecewise models that together define the behavior space 
of interest, with transition conditions between pairs of modes, ai and Oi+i given 
by the function 

: CaiX + DaiU > 0. (3) 

Model reduction techniques, such as singular perturbation and eigenvalue 
based techniques, are readily applicable to the linearized systems. They provide 
systematic methodologies to reduce the order of each piecewise model. Applying 
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singular perturbation, a small parameter, e, is removed from the model by letting 
its value tend to 0. This requires the formulation 



fat : X = Aai{e)x + Bai{e)u 

: Cai(e)x + Dai(e)u > 0. 



(4) 



In this formulation, slow and fast variables can be separated according to 

f .fy = ^lM)x + By.{e)u 
■ \ei = + 

Making e — > 0 leads to equations of the form z = f{y,u). Assuming that the 
system of algebraic equations is non singular, z can be substituted in the equation 
for y to derive an explicit reduced order ODE system. However, if e ^ 0 leads to 
a singular solution, 0 = f{y,u), system behavior is now defined by an implicit 
system of differential and algebraic equations (DAE), and the variable vector y 
may also include a fast component. In the limit, this fast behavior is replaced 
by an instantaneous projection {y, u), where y’*' is the initial value in 

mode tti+i, yai+i = Eind y is the value of the reduced order system in mode 
ai when > 0 was first satisfied. 

Similarly, when the system of equations becomes singular for e ^ 0, a state 
vector transformation may be required to achieve the desired separation and 
this may require a projection, ga{x,u). We discuss this in greater detail in Sec- 
tion 5. In general, it may be difficult to derive the transformation by analytic 
methods. Information about the physical system can be invoked to assist in de- 
riving the solution. The projection can be found by boundary behavior analysis 
of the detailed model, i.e., with the e parameter. As an alternative, or if this 
detailed model is not available, the projection can be computed by integrating 
the instantaneous field dynamics [4] and by subspace iteration [7]. These are 
implementations based on the use of the reducing subspaces of the Kronecker 
Canonical Form [2] to capture the state projection. 

The resulting model contains the reduced order specification of continuous 
behavior, /, the transition conditions, 7, and the projection equations, y. 






y = A'a^y + B'^.u 

c'y + D'u > 0 



' = E 



'ai+i2/ + 



Pcti+iU 



( 6 ) 



We study the effects of the order reduction technique on the state vector 
transfer function and transition conditions in this paper. Detailed analysis may 
be required when variables that constitute the 7 function exhibit impulsive be- 
havior. To identify such behavior, 7 can be expressed in terms of y. If any of 
the variables in the y vector are part of the algebraic constraints that develop 
when e ^ 0, they produce impulses. Detailed study may reveal the need for an 
additional transition modes to be introduced in the mode transition behavior. 
This transitional mode exists only at a point in time, and has no specification 
for continuous behavior. Furthermore, some transitional modes may have no ef- 
fect on the state vector. In previous work, we termed these transitional modes 
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pinnacles and mythical modes, respectively. A phase space analysis conducted 
in this paper establishes the relation between this approach and our established 
ontology for phase space transition behavior [ 6 ]. 

3 A Piecewise Model 

In a nonlinear continuous ODE model of the hydraulic actuator, the nonlinear 
characteristics of the externally controlled valve and the relief valve can be mod- 
eled as shown in Fig. 3 . Including the oil parameters (i?2 and C) results in the 
fifth order nonlinear ODE 
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(7) 

with Di = /i(i?i(si) -|- R2), D2 = l2{Ri{si) R2), and = C*(i?i(si) -I- i?2)- 
The variable Uy is externally controlled, and Ur = represents a 

function that approaches a step when a — > 00. The two state variables, si and S2, 
provide a parametric representation model for the detailed continuous switching 
behavior of the two valves. The cylinder oil pressure, p, expressed in terms of 
the state variables, is: 

P= p p (Pi + - R 2 {fi + / 2 )). ( 8 ) 

-Til H- Xl2 III 

When p approaches ipth, Ur becomes positive and the valve opens by switching 
to another behavior dimension (s2 > 0 ). Since Ur is always positive, the valve 
does not close, once it is opened. Therefore, transitions from ooi to aoo and aio 
are not defined in Table 2 . For the same reason, there are no transitions from 
Oil to oqo and oiq. 

Piecewise linearization of i?i(si) and Rs{p, S2) into regions of high resistance, 
Ri^hi and low resistance, Ri^i, is defined as: 

R\ = if {uy < 0 ) then Ri^h else R\^i . s 

i?3 = if (i?3 = i?3,/ or p > Pth) then i?3,/ else R^^h 

This allows for removal of the states si and S2 from the system model, resulting 
in a linear ODE model with four global modes: ooo ^ {Ri,hiR3,h}i aoi ^ 
{Ri^h, 7 ? 3 ,j}, oio ^ {Ri,h R3,h}, and an {Ri,i, R3,i}- The transitions between 
the modes are specified in Table 2 . 

4 Prom Complex to Simpler ODEs 

The parameters Ri^h and R^^h in the piecewise models are large compared to the 
other system parameters. The singular perturbation approach can be applied to 
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Fig. 3. Nonlinear valve resistance characteristics. 
Table 2. Mode transition table. 
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remove these large parameters > oo and oo) and arrive at simpler 

reduced order ODEs for each mode. To simplify notation, we set R\ = R\^i and 
R?, — Rz,i- The dynamic behavior in ooi is derived by R\^h oo and can be 
expressed as: 





r/ii 




1 

1 




r/ii 


faoi ■ 


/2 

.Pi. 


— 


H-?. 1 

If 

L C C ^ J 




/2 

.Pi. 



From Table 2 it is clear that this abstraction does not affect the switching con- 
straints that define further discrete transitions out of this mode. The only tran- 
sition out of this mode, Ooi — > oii, is governed by the external variable Uy 
{Uy > 0). 

For oio, — > oo implies fy = 0 and behavior reduces to a second order 

system 
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( 11 ) 



In both modes, the pressure, p, in the switching condition is given by Eq. (8). 
The reduced behavior in Ooo is given by an autonomous second order system 
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Introducing Ri^h oo into Eq. (8), results in this pressure, p, being expressed 
as 

p = Pi - R2fi > Pth- (13) 

It turns out that the spread of the eigenvalues in these linearized, simplified, 
and reduced systems of equations is still quite large. For example, given param- 
eter values, Rij = R^^i = 0.01, i?2 = 100, C = 5 • 10“®, R = 1, and R = 0.01, 
one of the eigenvalues is computed to be five orders of magnitude less than the 
others in the modes ooi and oiq. This implies that the system still operates at 
two widely differing time scales, and it may be possible to simplify the system 
model further by abstracting the i?2 and C parameters. Applying this change 
will affect the state variable pi, which is part of the switching condition, p > pth- 
This requires a detailed study of the switching characteristics. 

5 The State Mapping 

The application of singular perturbation methods to the model in the last section 
with and C as the small parameters, replaces some differential equations by 
algebraic constraints. For example, the agi mode is reduced from a 3'"'^ order to a 
1®* order system, whereas mode ooo is reduced from a 2"^^ to a 0*^ order (purely 
algebraic) system. This may cause state variable values to change discontinuously 
during mode transitions. 



5.1 Jump into Mode aoi 

When i?2 — *■ oo, Eq. (10) becomes a singular system of equations with — /i — /2 = 
0. In phase space, this algebraic relation constitutes a manifold to which behavior 
is confined. The dynamic system behavior on this manifold is derived by applying 
a transformation, x = I\fi — Rf2, which gives x = A3/2. Substituting for 
/i(= — 72) in the expression for x and eliminating /2 yields 



X = — 



R3 

h+h'"' 



(14) 



If mode aoi is entered at a point not on this manifold, an instantaneous pro- 
jection in the impulse space has to be executed to satisfy the manifold con- 
straint. The impulse space can be derived by integrating the dynamic behavior 
in Eq. (14) over an infinitesimal interval from t to T*", which gives Ii{f^ — /i) — 
Rif} ~ h) =0 [4]. Combined with the manifold constraint at C*", = — ft ^ 

this computes the projection to be 

9o.,.--ft = J^{hh-hh). (15) 

J^i + ^2 



Table 2 shows that the transition conditions for ooi are not affected by the 
variables /i and /2, therefore, no further analysis is required. 
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5.2 The Jump into Mode ctoo 

In mode ooo, R 2 ^ oo produces /i = 0. Again, this constitutes a manifold in 
phase space and transition into ooo requires a projection, 

5aoo : fi = 0- (16) 



However, analysis of the detailed model indicates that the switching condition 
from mode ooo to mode ooi in Eq- (13) may be activated before fi becomes 0. 
Therefore, this transition condition needs to be analyzed more precisely. 

When i ?2 ^ 00 , the switching condition in Eq. (13) becomes singular, and 
the value for p cannot be determined from the state variables. Therefore, p has 
to be expressed in terms of the time derivatives of the states. For this system, 
Eqs. (12) and (13) yield, p = The /i = 0 constraint corresponds to a 

discontinuous change in /i, therefore, fi may produce an impulse. 

Impulse behavior is too coarse an approximation of the underlying detailed 
continuous transient. A more refined analysis solves the detailed differential equa- 
tion in the time domain. The characteristic polynomial of faoo two roots 



A 1.2 = ^(-^2 ± (17) 

Assuming complex eigenvalues^ (Ai ,2 = K ± jAi), the pressure variable in mode 
aoo is (to = 0 for notational convenience) 

pi{t) = e^’'*(pi(0)cos(Ait) -f ^(pi(0) - XrPi{0))sin{Xtt)). (18) 

Applying a third order Taylor series approximation yields (pi = — ^/i), 

p,{t) = (1 + Xrt + ^)(pi(0)(l - ^) + (-^ - XrPmn (19) 

The switching condition is based on p = pi — i? 2 /i(t), where /i(t) is used instead 
of /i(0) because the value of fi changes during the time interval in which pi(t) 
rises and it may be different from /i(0) when p{t) reaches pth- 

This condition can be used to check if p > pth, and if so, the time, ts = 
ft{pi, fi,Pth), at which this constraint becomes true. This value can then be 
used in the expression for /i to derive the discontinuous change upon switching. 
Abbreviating /i(0) and pi(0) as /i and pi, respectively, and using a = (— — 
_|_ Ari? 2 /i)/Ai, b = p\ — i? 2 /i, and c = —pth, the solution is 

given by 

-(6-c) lc(A^ + Af)(6-c)^ 

^ aXi + cXr {aXi + cXr)^ 

Substituting ts in the expression for fi (t) in aoo results in the state mapping 



1 



9p,aoo ■ fi~ = e^'-*‘{ficos{Xits) + (-^/i + ^ 
Analysis of real eigenvalues is similar. 






(21) 
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where /i = /i(io) and the value of /i when ooo is exited because p > pth- 
This is graphically depicted in Fig. 4 for a third and fourth order Taylor 
approximation of /i, /f’^ and /f’^, respectively.^ Here an initial positioning 
maneuver of the piston is aborted at to> which causes the relief valve to open at 
ts ■ The error between the third and fourth order approximations is shown by ea 
and C 4 , respectively. 




Fig. 4. Value of fi at tg for a detailed model and its predictions at to- 



5.3 A Computational Model 

The discrete transition model that results from the abstractions of the detailed 
continuous behavior can be modeled by the extended hybrid automata structure 
in Fig. 5. The traditional hybrid automata is extended by junctions (indicated 
by small circles). When an event triggers a transition to a junction, the events 
on each of the exiting transitions from the junction are evaluated, resulting in 
an immediate second transition. 

In this model, when the external control valve closes {uy < 0), the time 
ts at which the relief valve opens is computed by ft{pi, fi^Pth) using the de- 
tailed continuous transient. If this computation returns a value tg > 0, con- 
trol is switched to the lower branch, else control switches to the branch at the 
right. This last branch indicates that the system moves to the field description 
for ckooj and, therefore, requires a consistent projection of the state variables 
(i.e., /i = 0). If the lower branch is taken, first the effect of the quick pressure 
build-up and corresponding flow decrease has to be accounted for by executing 
fi = 5p,aoo(/iiPij ^s)- This results in a new value for fi when the continuous 
behavior in ooi is activated. Again, behavior in this mode is subject to mani- 
fold constraints, and the corresponding projection fi = 5 aoi(/i>/ 2 ) takes place 

^ The predictions are computed during a short time interval around to to avoid sin- 
gularities that exist over the entire range. Note that the values only need to be 
computed at to. 
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u. < 0 / ELSE / 




fi = gaoiCfnfz) 

Fig. 5. Complex discrete switching structure. 



before ooi is activated to ensure values are consistent in this mode. Note that 
in aoi, Pi is not a state variable but derived from pi = pin — fiRi- Further, 
the systematically derived control structure is more complex as compared to the 
transitions in Table 2. 



6 Phase Space Transition Behavior 

The mode and discontinuous state changes can now be characterized in terms 
of a phase space transition ontology. In other work [6] , three principal transition 
functions were analyzed in phase space: (i) transition to a mythical mode, (ii) 
transition to a pinnacle, and (iii) transition to a continuous mode. 

When switching to aooi the two possible scenarios are 

1. p < Pth in which case a projection of fi onto /i = 0 occurs, and the system 
remains in Ooo. This represents a transition to a continuous mode. 

2. p > Pth in which case 

— there may be a distinct drop in /i before switching to ooi- This is a 
transition to a pinnacle, or 

— the switch to ooi has occurred before any significant change in fi occurs. 
This represents a transition to a mythical mode. 

The switch to ooi may also include a discontinuous state change because of the 
manifold projection that immediately follows the pinnacle or mythical mode. 
Figure 6 shows the phase space transition behavior for two values of C in a 
hybrid model with parameter values Rij = Rsj = 0.01, Ri^h = Rs,h = ITO^, 
i ?2 = 100, Ii — 1, I 2 = 0.01, and pth = 1000. Velocity /i is plotted on the x-axis 
and pressure p is plotted on the y-axis. The discontinuous approximations are 
superimposed by dotted lines. ^ When the control valve closes, fi has value 4, 
and the pressure in the cylinder starts to rise quickly (Fig. 4 depicts the time 
domain behavior). This behavior consists of an immediate change in p caused 
by the term and a quick continuous change because of the pressure build 

up. 

® These approximations are not simulation results. 
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(a) C = 1 • 10"® 



(b) C = 1 • 10“® 



Fig. 6. Dominant C phase space switching behavior. 



If the absolute pressure exceeds 1000, the mode switch to ooi occurs. In this 
mode, there is another quick change in /i, this time governed by the dependency 
between I\ and I 2 ■ Because /i is several orders of magnitude larger than I 2 , only a 
small change in fi occurs. The interaction between the three state variables in the 
C'’ hybrid model, /i , / 2 , and pi , causes oscillatory behavior in agi • This is clearly 
seen in Fig. 6(b). The discontinuous jump does not include this behavior, but 
immediately reaches the final value. The phase space behaviors present examples 
of two consecutive discontinuous state variable value changes that are of different 
types. The intermediate value of /i is achieved in a pinnacle mode, and the final 
value is governed by a manifold projection. Note that the pinnacle is crucial 
in computing the correct final value of the variable in aoi, when continuous 
behavior resumes. 

If i ?2 > 250, /ii ?2 becomes the dominant factor in the phase space transition 
behavior, as shown in Fig. 7 for C = I • 10“® and R 2 = 500. The consecutive 
switch to ooi follows immediately after the switch to aoo- As a consequence, 
aoo does not affect the value of /i, therefore, this is a mythical mode. Mode 
aoo is not intrinsically a mythical mode because the state variable values when 
the mode is entered determine whether it is exited immediately. Only in such 
situations mythical behavior occurs. The projection in aoi that follows is shown 
more clearly in Fig. 7(b) for a larger value of l 2 - For these parameters, Eq. (15) 
verifies the value /(*" = 2 (/i = 4, /2 = 0) and confirms that larger values of 
I 2 have a greater effect on the magnitude change of /i. Again, the fast oscilla- 
tory behavior of the manifold projection is abstracted away in the discontinuous 
approximation . 

7 Conclusions 

This paper shows how nonlinear and high order system models can be system- 
atically reduced to piecewise linear systems with more uniform time scales of 
behavior. The resultant hybrid model is obtained in two steps: (i) continu- 
ous with piecewise simpler behavior and switching conditions, and (ii) piecewise 
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(a) h — 0.1 



(b) h = 1 



Fig. 7. Dominant i ?2 phase space switching behavior. 



continuous reduced order behavior with switching conditions and discontinuous 
changes in state variable values. The reduction in continuous domain complex- 
ity is gained at the cost of increasingly complex discrete event control struc- 
tures. Because of the intricacies in defining the switching conditions and the 
corresponding jumps in the variable values, ad hoc modeling schemes can often 
produce erroneous results. This is most likely to happen when jumps occur in 
state variable values, caused by the introduction of algebraic constraints. The 
manifold projections that result may be aborted because intermediate variable 
values derived from the detailed dynamic models indicate that further immedi- 
ate transitions occur. These concepts are illustrated by analysis of phase space 
behavior of a hydraulic actuator. 

The approach fits into our ontology for describing transition behavior in phase 
spaces that we have established in previous work [6]. We hope to extend this 
approach to systematic procedures for automated model reduction of complex 
nonlinear systems into simpler hybrid representations. A longer term goal of 
this work is to develop real time models of complex systems so that they may 
be employed in hybrid observers for Fault Detection and Isolation (FDI) studies 
of complex nonlinear systems [5] . 
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Abstract. We propose a computer-aided methodology to automatically 
generate time optimal production schemes for chemical batch plants op- 
erating in multi-batch mode. Our approach is based on the following 
principles: (1) the plant is modeled at the level of process operations 
whose behavior is specified by timed automata, (2) the optimal produc- 
tion schemes are generated using algorithms for reachability analysis of 
timed automata implemented in OpenKronos, (3) the output of the 
verification tool is post-processed to derive high-level control code. We 
apply our methodology to the batch plant at the University of Dort- 
mund. The automatically computed operation schemes turned out to be 
more efficient than the previously used handwritten ones. 



1 Introduction 

A chemical batch plant consists of a collection of containers, reactors, pipes, 
valves, pumps, etc., for storing, transporting, processing and transforming raw 
materials to obtain a final chemical product. A plant is also equipped with an 
integrated hardware and software architecture for controlling and supervising 
its operation. Generally, batch plants are operated in multi-batch mode where 
several products are manufactured concurrently. The structure and operation 
of batch plants are standardized in the norm ISA S88.01 [7]. A central notion 
of the ISA S88.0I standard is that of a plant-independent recipe, a description 
of “abstract” processing steps (e.g., mixing, heating, cooling) leading to a pro- 
duction goal. For a specific batch plant, it is the task of the control engineer 
to construct an operation scheme, that is, a “concrete” arrangement of process 
operations of the plant (e.g., mixing materials A and B in container C, emptying 
out container C into container D, cooling the content of container C), that realize 
a given plant-independent recipe. Process operations are actually carried out by 
sequences of low-level process actions (e.g., opening and closing valves, starting 
and stopping pumps), which are commanded by hardware or software that im- 
plements each process operation as a procedure that is invoked by a high-level 
control program. 

* This work has been supported by EG Esprit-LTR Project 26270 VHS. 
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An operation scheme should make an efficient use of the resources and satisfy 
all the constraints required for a safe and correct functioning of the plant. The 
usual approach followed by the control engineer to find an optimal operation 
scheme out of a plant-independent recipe is somewhat analogous to the one fol- 
lowed by the cook to make a cake out of a recipe found in a cookbook. Roughly, 
it consists in first providing a plant- dependent recipe made up of a (partially) 
ordered set of process operations of the plant that can be done to realize the 
plant-independent one. The plant-dependent recipe specifies a set of possible, 
though eventually conflicting and not necessarily optimal, sequences of opera- 
tions, together with constraints on the usage of shared resources. In the terms of 
our analogy with the cook, this step corresponds to determining which particu- 
lar kitchen-ware to use and how to use it (e.g., for how long, how many times, 
under which conditions, when, to do what, etc.) in order to realize the different 
actions specified in the recipe. The second step consists in finding an optimal se- 
quence that meets the constraints. Typically, the plant-dependent recipe is given 
as an acyclic directed graph. Standard (combinatorial) optimization techniques 
are used to find the best path in the graph. Notice that such sequence may not 
exist, which indeed means that the plant-dependent recipe cannot be realized by 
that operation scheme. Again, this is similar to what the cook does. She figures 
out that certain steps of the recipe can be done concurrently, e.g., cooking the 
cake in the oven and preparing the chocolate sauce to cover the cake, that oth- 
ers are conflicting due to the existence of limited resources, e.g., preparing the 
paste and the chantilly, and eventually that the cake cannot be made at all, e.g., 
because the chosen oven does not heat enough. In this approach, much of the 
intrinsic complexity of the problem needs to be taken care of by the engineer (or 
the cook) during the specification of the plant-dependent recipe. However, even 
for a single batch, constructing that recipe is not a trivial task and it becomes 
significantly more difficult for multi-batch processing (i.e., making several cakes 
concurrently on the same kitchen), specially for complex batch plants exhibiting 
a high degree of parallelism. 

The first aim of this work is to alleviate the task of the engineer, at the 
cost of eventually using computationally more expensive techniques, by provid- 
ing computer-aided support to automatically generate an operation scheme for 
multi-hatch processing, without having to specify a plant-dependent recipe, but 
only a mapping between “abstract” processing steps and production goals into 
one or more “concrete” process operations that can effectively realize them. The 
second objective is to automatically derive the control program that carries out, 
in real time, the operation scheme on the control architecture of the plant. 

The basic underlying idea to do so is to require the engineer to provide an 
“operational” model of the plant, together with (1) the production goal, which 
may consists of an arbitrary number of batches, and (2) the optimization crite- 
rion, typically shortest overall production time. The major difficulty that arises 
here concerns the modeling of the operation of the plant at a level of abstraction 
suitable for both the recipe and the control architecture (hardware and soft- 
ware) of the plant. In general, such a modeling framework would need to take 
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into account both the discrete events and the continuous chemical and physical 
phenomena, leading to the need of using a hybrid model, e.g., hybrid automata 
[1]. However, if we consider the problem at the level of process operations, it 
is possible to use simpler models, e.g., timed automata [2], that abstract away 
most of the details of the complex continuous behaviors, while preserving all 
timing and concurrency constraints relevant to the operation of the plant. This 
is mainly due to the fact that: (1) many recipes are indeed described in terms of 
quantities of raw materials and timing constraints on abstract processing steps 
such as mixing, heating, cooling, etc., and (2) the execution time of process 
operations can, generally, be estimated quite accurately. 

In order to achieve our goal, we propose a methodology based on the following 
principles, models and tools: 

1. The plant is modeled at the level of process operations whose behaviors are 
specified by timed automata extended with shared variables. 

2. The optimal production schemes are generated using the algorithms for 
reachability analysis of timed automata implemented in the VERIMAG tim- 
ing verification toolsuite OpenKronos [5]. 

3. The output of the verification tool is post-processed (1) to visualize the 
operation schemes in different ways (e.g., Gantt and Hasse diagrams), and 
(2) to derive high-level control code. 

In order to illustrate the feasibility of the approach in practice, we apply it to a 
case study: The chemical batch plant of the Dortmund Process Gontrol Labora- 
tory [11]. We derive time- and resource optimal schedules for several number of 
batches. Moreover, the operation schemes computed by the tool turned out to 
be more efficient than the ones obtained using the “classical” cook-like method 
described before. 

The rest of the paper is structured as follows. In Section 2 we sketch a frame- 
work for modeling the operation of batch plants at the level of abstraction of 
process operations. In Section 3 we describe the case study. In Section 4 we 
present our approach for searching for optimal operation schemes using reacha- 
bility analysis. In Section 5 we report on the experimental results obtained for 
the case study. In Section 6 we discuss current work concerning the integration 
of the approach into the control architecture of the plant. 

2 Modeling Chemical Batch Plants 

We model a plant by specifying: 

— A collection of resources. In principle every single valve, tube and container 
may be involved in the execution of process operations, but there are many 
devices which are reasonably used together^, e.g., a valve is only used in 
combination with its containing pipe. As a rule, the most important devices 
are the containers. In almost all cases, the surrounding pipes of a container 

^ These groups are called units in the standard. 
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are only used in combination with transfers from or to this container. Thus, 
a typical resource would be a container. Also, we can abstract away those 
resources which are only used in combination with other (modeled) resources 
and do not contribute to the state of the system on a macroscopic level. 

— A collection of possible discrete contents of each resource (typically con- 
cerning mass or volume, temperature, chemical phases ... ) . The process 
operations are assumed to perform discrete transitions on the state space 
of container contents. Hence, it is usually possible to give a discrete and fi- 
nite representation of the contents of a container as it occurs before or after 
the execution of process operations. In particular the possible values must 
encode intermediate products in recipes to allow to map recipes onto the 
plant. 

— A collection of process operations. Each process operation is furthermore 
associated with: 

• A name (the name of a PLC control routine). 

• A collection of resources used by the operation. 

• A condition for the enabledness of the operation, which depends on the 
states of the involved resources. 

• A function representing the transition on the states of the involved re- 
sources. 

• A function to estimate the time consumption of the operation on the 
bases of the container states before the operation. 

Based on this semi-formal description, a formal specification is derived as 
a network of communicating timed automata [2] appropriately extended with 
shared variables. To obtain the formal model we proceed as follows: 

~ The availability of each resource is modeled by a boolean variable. 

— The contents of resources are correspondingly modeled by shared variables 
(volume, temperature, . . . ) over finite domains. 

— Each operation is realized by a timed automaton which has (in addition to 
the shared variables) two control locations non-active and active, as well as 
a start transition and a finish transition. 

• The start transition which depends on the guard for the operation and 
the availability of resources, reserves the resources and starts the clock. 

• The finish transition which depends on the duration constraints on the 
clock, changes the values of the variables modeling the content of the 
containers, and releases the resources. 

• Invariant constraints associated with control locations guarantee that 
transitions must occur within the predefined time bounds. 

The reader is referred to [2] for a detailed description of the formal semantics 
of the timed automaton model. In the next section we informally present the 
semantics through an illustrative example. 
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3 Case Study: Modeling 

We model here the chemical batch plant from the Dortmund Process Control 
Laboratory [11]. An overview of the plant and the architecture of the integrated 
control system is given in Figure 1. 




remote 

computing 

server 



real time model 
on process operation level 

L 



operation 

scheme 






Windows PC with GUI and 
high level eontrol 



Fig. 1. Control architecture of the plant. 



The plant consists of seven containers, namely B1 to B7. Containers Bl, B2 
and B4 are ordinary ones. Container B3 has a device for mixing. Container B5 
is the evaporator connected to a condenser. The condensed steam flows into 
container B6. Both B6 and B7 are attached to a cooling system. 

There are essentially three levels in the control. The lowest level concerns the 
physical control elements, such as sensors, valves and electric devices for pump- 
ing, heating and stirring. On top of this, there exist a number of basic control 
routines implemented on a Siemens S7 PLC (and described by SFCs, sequential 
function charts, in [11]), which realise process operations. These control routines 
are invoked from the higher layer by an operator or by a control program running 
on a PC. 

The plant-independent recipe is as follows: 

1. Produce highly concentrated brine by manually adding salt to tap water. 

2. Mix it with demineralised or tap water to produce a medium concentration. 
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Table 1. Process operations of the plant. 



operation 


description 


duration 


B2 


fill B2 with water from tap 


lOs/1 




Fill 41 of pure water into B3 


320s 




Manually add NaCL into B3 and mix until concentration is 5g/l 


600s 




Pump concentrated solution from B3 to Bl 


420s 


B3A 


Fill 41 of concentrated brine from Bl into B3 


320s 


B3B 


Thin down concentration in B3 to 3g/l 


240s 


B3B4 


Fill solution from B3 into B4 


600s 


B4B5 


Fill solution from B4 into B5 


330s 




Evaporate and condensate from B5 to B6 until high cone, reached 


1500s 




Fill hot concentrate from B5 to B7 


260s 


B7 


Cool solution in B7 


600s 




Pump up solution from B7 to Bl 


220s 


B6B2 


Pump up pure water from B6 to B2 


240s 



3. Heat and evaporate water out of this medium solution such as to return to 
the high concentration and condensate the vapor and capture the condensate 
(demineralised water). 

4. Cool down the resulting solution. 

A batch is finished when the highly concentrated remainder after evaporation is 
cooled down. 

The actual modeling follows precisely the scheme indicated in Section 2. Our 
model allows for the maximal exploitation of parallelism in the plant, which is 
of great importance for efficient multi-batch execution. The process operations 
are listed in Table 1. The duration estimations are derived from experimental 
values listed in [11]. ^ Due to lack of space, we do not present here the full 
model, but focus on the description of the variables that have been used and the 
specification of some of the more illustrative process operations. 

For each container Bz there is (1) a boolean variable Bz that models the 
availability of the container, and (2) a discrete variable Vi ranging over a finite 
domain, modeling the relevant values of volumes of liquid in the container, e.g., 
0 (empty) and 4 (41) for Bl, and the interval of values from 0 to 6 for container 
B2. For container B3, there is a discrete variable C3, modeling the significant 
values of the concentration, namely Og/1 (demineralized), 3g/l (medium) and 
5g/l (high). For container B7, the variable H7 is used to model the two possible 
estimations of the temperature of its content, namely hot and cold. In order to 
determine the number of finished batches, we use the additional variable count, 
ranging over the natural numbers. 

^ For the sake of simplicity, operations for occasional rinsing, which do not contribnte 
to the production, have been omitted. Furthermore, we have split some operations 
involving B3 into two parts to introduce more potential parallelism into the system. 
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Table 2. Specification of the containers. 



container 


volume 


temperature 


concentration 


B1 


VI 


{0,4} 






B2 


V2 


[0,6] 






B3 


V3 


{0,4, 7} 




C3 : {demineralized, medium, high} 


B4 


V4 


[0,7] 






B5 


V5 


{0,4, 7} 






B6 


V6 


{0,3} 






B7 


V7 


{0,4} 


H7 : {hot, cold} 





des(0,2,2) 

(0, [B7 A H7=hot] B7start B7:=false CL7:=0, 1) 

(1, [CL7=60] B7finish B7:=true H7:=cold count : =count+l , 0) 



Fig. 2. Timed automaton modeling process operation B7. 



The timed automaton modeling the operation of cooling the content of B7 
is depicted^ in Figure 2. The automaton has two control locations, namely 0, 
the initial one, and 1, and two transitions, labeled B7start and B7finish. The 
guard of B7start checks if the container B7 is available and hot. In such case, the 
transition is said to be enabled and can be executed. When doing so, container 
B7 is reserved for exclusive use, by setting B7 to false, the clock CL7 is reset to 0 
to start measuring the duration of the cooling process, and the automaton moves 
to location 1. In this location, the automaton waits until the corresponding clock 
has reached 60, modeling the 600s of cooling, releases B7 by setting B7 to true, 
changes H7 to cold, and moves back to location 0. Transition B7finish indeed 
models the completion of a batch, which consists in obtaining a cold, highly 
concentrated brine in container B7. Therefore, the value of count, representing 
the number of already produced batches, is updated when B7f inish is executed. 



des(0,2,2) 

(0, [B5 A B7 A V5=4 A V7=0] B5B7start B5:=false B7:=false CL5:=0, 1) 
(1, [CL5=26] B5B7finish B5:=true B7:=true V5:=0 V7:=4 H7:=true, 0) 



Fig. 3. Timed automaton modeling process operation B5B7. 



The automaton modeling the operation of emptying out the content of B5 
into B7 is depicted in Figure 3. It has two control locations and two transitions. 

® The complete syntax of the input language of OpenKronos can be found in 
http : //www-verimag. imag.fr/DIST_SYS/SMI. 
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In location 0, it waits for B5 and B7 to be free, the volume of B5 to be 41, and 
B7 to be empty. In such case, it can move to location 1, while blocking the use 
of both containers, and resetting the clock CL5 to start counting the time spent 
in the operation. When the value of CL5 reaches 26 (modeling the 260s required 
for emptying out B5 into B7), the automaton moves back to location 0, releases 
the containers, and updates the values of the variables modeling the contents 
(V5 and V7 become 0 and 7, respectively). 

4 Synthesis of Optimal Operation Schemes 

The problem we are interested in solving is the following: Given a specification 
of a batch-processing plant as a set O of process operations and a number N 
of (identical) batches to be produced, find an operation scheme, i.e., a partial 
order of process operations tt = (O', <) where O' is a multiset of elements 
of O (allowing for multiple occurrances of the same operation), that executes 
the required N batches. Notice that we search for a partial order of operation 
instances that allows for parallel execution of independent operations. Besides, 
the operation scheme tt is also required to satisfy some “optimality” criteria 
related to the time spent to finish the N batches, the number of resources used, 
etc. 

At first glance, this problem can be viewed as a particular instance of the 
more general problem of “controller synthesis” stated as follows: Given a plant P 
and a property S, construct a controller C that “forces” P to meet S by disabling 
some (controllable) behaviors of P. A technique for solving this problem in the 
context of discrete-event systems has been first proposed in [14]. This technique 
has recently been extended to timed systems in [10], but the currently available 
prototype [3] is not able to deal with large systems like the one we are considering 
here. 

Fortunately, model-checking provides us with means to look at the problem 
from another angle. Indeed, since the execution times (more precisely, the upper 
bounds of the execution times) of the process operations are known and the 
operation scheme to be calculated is finite, a solution to the problem can be ob- 
tained by using a reachability algorithm capable of providing a (timed) sequence 
of start and stop transitions that reaches the desired goal. 

Gertainly, any “standard” reachability algorithm will allow us to find some 
operation scheme (if at least one exists) but not necessarily an optimal one. 
An algorithm for solving the “optimal-controller synthesis” problem has been 
recently proposed in [4], but not yet implemented. However, a much simpler 
“ad-hoc” solution can be devised by making use of the knowledge we have about 
(1) the particular search method used by the reachability algorithm and (2) the 
structure of the plant. 

Using a breadth-first exploration of the reachability graph ensures that the 
operation scheme found makes optimal use of the resources, in the sense that 
no other operation scheme can achieve the same goal executing fewer process 
operations. However, the operation scheme might not be the fastest, that is, there 
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might be another one that performs the same number of process operations in less 
time. One possibility to overcome this problem is to guess an upper bound T for 
the completion time and to iterate the reachability exploration by appropriately 
increasing or decreasing the time horizon according to the result obtained in the 
previous iteration. By applying this strategy, the optimal operation scheme can 
be obtained in log(T) number of iterations. 

Still, the size of the state-space to be explored might be a serious obstacle 
and it is advisable to exploit the knowledge of the plant to try to overcome it. 
For example, in the case study, it is easy to see that optimal operation schemes 
must intensively use resource B5. Appropriate use of this information has indeed 
revealed to be vital for solving the problem. 

This approach can be automated using state-of-the-art verification tools for 
timed automata, such as OpenKronos [5]. In particular, for the case study we 
have used a discrete-time BDD-based reachability algorithm The result of the 
timing analysis is a timed trace, which is a sequence consisting of ticks, repre- 
senting the elapsed time units, and of transitions of the automaton, representing 
the beginning (e.g., B7start) and termination (e.g., B7f inish) of process oper- 
ations. 

Such a trace can be visualized as a Gantt-like diagram, where the operation 
instances are visualized as blocks in a two dimensional diagram, one dimension 
for the resources (blocked by the operation), the other for time. Notice that in 
this context some operations may use several resources. 

However, we are looking for a partial order of operations, which we can 
reconstruct out of the timed trace: By specification, two instances of process 
operations a and b using a common resource R will not be active in parallel, i.e. 
if we find the start event of a in the trace before the start event of b, then also 
the finish event of a comes before the start event of b. In this case we say that 
operation instance b depends on operation instance a (a <' b). The transitive 
closure of these dependencies between arbitrary occurrences of operations in the 
scheme gives us a partial order, the operation scheme. 

The actual output of this algorithm is the Hasse diagram of the partial order, 
without any timing information. This partial order serves as input to a high-level 
control software, as will be explained in Section 6. We consider the temporal 
information here only as justification, why this partial order of operations is 
realisable under the timing constraints imposed by the specification. No timing 
is needed for control here. 



5 Case Study: Synthesis 

The plant-dependent recipe for the production of cold, highly concentrated brine 
described in [11] (and designed for manual operation by students) consists of two 
phases: 

^ A BDD is a compact data structure for encoding boolean predicates (e.g., sets of 
states). The reader is referred to [9] for more details. 
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Fig. 4. The timed trace calculated for three batches as Gantt diagram. 



— A preparatory phase which aims to place highly concentrated brine in con- 
tainer B1 and water in B2: B2, B3KA, B3KB, B3U, B2. 

— A cyclic scheme to produce a single batch with the invariant that before and 
after this scheme there is concentrated brine in B1 and water in B2: B3A, 
B3B, B3B4, B4B5, EVAP, B5B7, B7 to obtain the final product, and finally 
execute B6B2, B7B1 to recycle material. 

Notice, that the recipe allows for very little parallelism for a single batch (only 
process operations B7 and B7B1 can be executed in parallel with B6B2 and, if 
the next batch is taken into account, B3A can be executed in parallel), under- 
exploiting the capacity of the plant. Obtaining a better performance requires 
using another recipe which cannot be obtained by re-scheduling the given one. 

Instead of trying to schedule several instances of this recepy, we had the 
computer search for operation schemes for the production of several batches (i.e. 
for observing operation B7 several times). 

Figure 4 and Figure 5 respectively show the Gantt-like and Basse® diagrams 
of the computed operation scheme for three batches. 

It is interesting to note that the calculated operation scheme requires less time 
and less operations than one described in [11]. First, the second B2 operation can 

The picture of the Basse diagram is obtained using the graphviz tool from Bell Labs. 
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Fig. 5. The operation scheme for three batches as Hasse diagram. 



be done concurrently with B3KB since they do not share any resource. Second, 
the sequence B3U, B3A, is actually never used. Indeed, this sequence is redundant 
because the concentrated brine is mixed in B3 and there is no need to pump it 
to B1 (operation B3U) to re-pump it later back into B3 (operation B3A), a cycle 
both taking a significant amount of time and making a non-efficient use of the 
resources. 

Another interesting observation is that, to achieve optimal performances with 
respect to time, it is sufficient to manually produce only two portions of concen- 
trated brine for the first two batches. The following ones will re-use the brine 
produced by the previous ones and stored in Bl. This is because the operations 
involving container B5, in particular the heating but also the re-fillings into and 
from B5, take long enough for the preparation of the next batch in B4 out of 
the already produced brine. This is a robust feature of the plant. Actually, we 
have experimented with an imaginary stronger heating system performing the 
evaporation in half the time with the same result. Evidently, the period in the 
optimal continuous operation of the plant is the sum of the times of the opera- 
tions involving B5. Hence, changes to the hardware affecting other parts of the 
system, such as a hose recently incorporated into B4 to speed up the refilling 
from B3 to B4 only speed up the production of the first batch. In contrast, any 
improvement on the three operations involving B5 will directly speed up the 
multi-batch production. 

Concerning the performances of the tool. Figure 6 shows the measured com- 
putation times and memory requirements of OpenKronos for a growing number 
of batches. Intermediate curves between the discrete values give an impression 
of the growth of requirements relatively to the search depth (i.e., number of 
batches) . These results were obtained with a 200MHz PentiumPro linux system 
with 500MB of main memory. With a growing number of batches (and thus a 
growing search depth) the memory and time consumption rises, but the size 
of the BDD representing the set of reached states stops growing after about 6 
batches and then constantly requires about 200MB of storage. The BDD of the 
whole reachable state space has about 400K nodes representing about 16 billion 
of states. For more than 6 batches the execution time nevertheless rises linearly 
with the number of batches required. For an experiment with 32 batches the 
calculation took about 9 hours. 
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Fig. 6. Time and space performances of OpenKronos. 





6 Integration into the Control Architecture of the Plant 

The last step towards an end-to-end deployment of the proposed methodology 
consists in being able to fed the computed optimal operation scheme into the 
control software. 

The high level control is realised by a dispatcher, which is in charge of trigger- 
ing process operations. Ideally, the dispatcher would start the process operations 
at the starting times figuring in the Gantt-diagram and the operations would 
finish at the given times. However, the times in the model are estimations and 
not neccessarily exact. In short, the dispatcher can control the beginning of an 
operation but not its end! 

Therefore, the dispatcher uses the operation scheme - the causal order - 
without any timing information and starts operations following the strategy as 
soon as possible: 

— Initially, the dispatcher invokes the process operations without predecessor 
in the partial order. 

— Afterwards it is called by events that indicate the termination of a process 
operation. Then the dispatcher removes that process operation from the 
partial order and triggers all those operations, which have no predecessor 
now. 

— When the partial order is empty, the dispatcher signals the completed pro- 
cessing of the operation scheme. 

The important properties of this strategy are: 

— Under the assumption that reality follows exactly the model in operation 
times, the operation scheme is actually optimal. 

— Under the assumption that the operation times given were safe upper bounds, 
also the whole process as controlled by the dispatcher satisfies the global time 
bound. 

— Even if an operation should take longer than expected, the process can con- 
tinue. It will just wait and on the whole possibly take longer than expected. 
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Of course, there are systems where this simple strategy does not always work. 
For instance, upper bounds on some operations may require to start a certain 
process operation with delay. 

The control architecture of the plant is depicted in Figure 1. The plant is 
equipped with a Siemens Step 7 control system consisting of an S7 PLC which 
is connected to a Windows PC. The PC runs Step 7 for development of PLC 
programs and WinCC, a PC based component in the control that serves for vi- 
sualization, data acquisition, as well as high-level control. WinCC communicates 
with the PLC over a communication line and is programmed to communicate 
with the PLC software via events concerning variables on the PLC. Process op- 
erations are implemented as PLC control programs that can be invoked from 
WinCC, either by manual user interaction (clicking on a button) or by WinCC 
routines written in C. The PLC routines in turn communicate to WinCC when 
they are finished. 

The operation scheme is fed to the dispatcher, which is realized as a C- 
callback function of the WinCC runtime system. On the whole, the operation is 
performed as follows. 

— From manual mode, the operator requests automatic operation from the 
GUI by pushing a corresponding button and chosing the desired operation 
scheme via a file dialogue. The system then starts the dispatcher and goes 
to automatic mode. 

— When the operation scheme is finished, the dispatcher returns control to 
manual mode. 

Safety requirements are met by allowing the operator to interrupt the dis- 
patcher and return to manual mode. Obviously, this action requires manually 
returning the system to a consistent state. 

7 Concluding Remarks 

We have proposed an approach to automatically generate high-level control code 
for multi-batch processing using timed automata and their associated reachabil- 
ity analysis tools. Certainly, timed automata and model-checking algorithms 
can also be used to verify the correctness of low-level control programs (PLC 
routines) as it is done for the Dortmund plant in [8,6]. 

We believe that the experimental results obtained with the case study are 
encouraging. They illustrate the applicability of the approach in practice. The 
“off the shelf” tool (OpenKronos) we used was chosen for mainly two reasons: 
(1) immediate availability and (2) natural support of the modeling framework. 
Of course, the methodology does not depend on this choice. Indeed, there may be 
specialized algorithms, which eventually perform better on the particular class 
of models used. For instance, since it is usually possible to predict a bound on 
the size of the operation scheme (in total time as well as number of operations), 
it is possible to state the problem as an NP problem and then to apply a SAT 
solver (e.g., [12]). 
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We are currently working on the integration of our methodology into the 
Dortmund plant in the context of the European Project ESPRIT-LTR VHS 

[13]. 
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Abstract. In this paper we propose a verification method for hybrid 
systems that is based on a successive elimination of the various sys- 
tem locations involved. Briefly, with each such elimination we compute a 
weakest precondition (strongest postcondition) on the predecessor (suc- 
cessor) locations such that the property to be proved cannot be vio- 
lated. Experiments show that this approach is particularly interesting 
in cases where a standard reachability analysis would require to travel 
often through some of the given system locations. 



1 Introduction 

Hybrid Systems are real-time systems that are embedded in analog environ- 
ments. They contain discrete and continuous components and interact with the 
physical world through sensors and actuators. A common model for hybrid sys- 
tems can be found in hybrid automata. These are finite graphs whose nodes 
correspond to global states as illustrated in the famous “Leaking Gas Burner” 
example [ACH+95]: 



Leak Non-Leak 




Nodes “Leak” and “Non-Leak” represent discrete locations, whereas x, y, and z 
are data variables. Each location may contain a location invariant (x < 1 in the 
example) and the continuous activity which describes how the values of the data 
variables change in time. In the above example the value of x and y increase by 
1 per time unit (say, second), i.e., the first derivative of the function describing 
the behavior of x and y over time is the constant 1. 2 also increases by one per 
second in location “Leak”, however, it remains unchanged (i = 0) in location 
“Non-Leak” . Edges are annotated with guards and discrete actions. Guards form 



N. Lynch and B. Krogh (Eds.): HSCC 2000, LNCS 1790, pp. 352—365, 2000. 
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a constraint on the data variables to hold if a transition via the corresponding 
edge is to be performed. The discrete action specifies how the data variables are 
to be changed after taking the transition. In the above example the guard of the 
edge from “Leak” to “Non-Leak” is logical truth (T), i.e., no special condition 
has to be fulfilled, whereas the guard of the edge from “Non-Leak” to “Leak” is 
X > 30, i.e., this edge may only be taken in case the value of the data variable 
X is at least 30. The discrete action for both edges is to reset a: to 0. 

A computation of such an automaton is a sequence of state changes (steps) . 
Within each step the system state evolves continuously according to a dynamical 
law until a transition from one node to another one occurs. 

Since hybrid systems typically operate in safety-critical situations, the de- 
velopment of rigorous analysis techniques is of high importance. In the last 
ten years several proposals for a verification methodology for hybrid systems 
arose [ACD90, ACH+95, ACHH93, AD94, AH92, AHH96, AHS96, ANKS95, 
GNRR93, Hen96, HNSY92]. Most of them are based on a so-called (forward 
or backward) reachability analysis. Intuitively, a forward reachability analysis 
(for safety properties) performs the following operation: starting from the initial 
situation (state), all possible (immediate) time and edge successor states are 
computed. Then the resulting set is reentered as an input to compute further 
time and edge successors, and so on. This will go on until no further new states 
can be derived (reached). Provided this procedure at all terminates, it ultimately 
comes up with the set of all reachable states (reachable from the initial state) 
which may be used to check the property to be proved. 

Backward reachability, on the other hand, starts from a description of the 
states that do not fulfill the property to be proved and tries to compute all 
possible predecessor states, i.e., all the states from which one of the unsafe states 
could possibly be reached. Again, upon termination, it ends up with a set of 
states all of whose elements may lead to an unsafe situation, and what remains 
to be done is to check whether the initial state is contained in this set or not. 

At the first glance, at least for forward reachability, and upon termination, 
it can hardly be seen that anything else could behave better. After all, forward 
reachability computes exactly the reachable states (and nothing else) and we 
need to know about all the reachable states in order to perform our verification 
task. Indeed, forward reachability does not compute any redundant information. 
However, it may perform redundant computations. For instance, if the property 
to be proven requires several passes through certain locations, the actual effect 
is usually very similar to earlier (or later) passes. Only the values of the variables 
involved might vary a bit, although in some more or less regular way. 

The purpose of the approach proposed in this paper is to show how to gain 
such a knowledge and how to take advantage of it, i.e., to compute the behavior 
of locations once and for all and to forget about these locations later on. This can 
result in certain extra properties to be proved for the other locations (weakest 
preconditions or strongest postconditions) that take over the responsibility of 
the location just eliminated. The following example illustrates this: 
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Li 




Let us assume that we want to prove that a; + y < 10 is an invariant of the system 
(from which we can only see a small portion in this illustration). It might very 
well be that any reachability analysis will have to go through location L2 several 
times and therefore we want to eliminate this very location by computing a 
weakest precondition on location L\ that guarantees that L2 could impossibly 
violate the desired property. The approach to be presented in this paper will come 
up with the following result: the invariant indeed holds for the whole system if 
and only if the invariant holds for the “simplified” system 




X < y \ X := 0;y := 0 




provided we can guarantee that x<y^ 2 y<x + 5 whenever we are within 
location L\. In what follows we describe formally how such a condition can be 
computed and how this method can be used as a verification tool for hybrid 
systems. 

2 Preliminaries 

Given a fixed variable set X we define the set CT of Constraint Terms (over the 
variable set X) as the smallest set containing the reals and the set X and that 
is closed under addition, subtraction and multiplication with reals. The set CF 
of Constraint Formulas (over the variable set X) is defined as the smallest set 
containing T, _L (logical truth and falsity respectively), ti > t2, ti > t2, t\ < t2, 
ti < t2, and ti = t2 (where ti and t2 are constraint terms), and that is closed 
under logical conjunction. 

A hybrid system is a tuple of the form H = (A, C, £, dif, inv, guard, act) where 
A is a finite set of real-valued data variables, £ is a finite set of locations, i.e., 
nodes of a graph, £ C £ x £ is a finite (multi)set of transitions, i.e., edges of the 
graph with nodes from £, dif: £ x X i—> CT is a mapping that associates with 
each location and each data variable a constraint term (over A), inv: £ 1— > CF is 
a mapping that associates with each location a constraint formula, representing 
the location invariant, guard: £ 1— > CF is a mapping that associates with each 
edge a constraint formula, representing the enabling condition for this transition, 
and, act: £xX ^ CT is a mapping that associates with each edge and each data 
variable a constraint term, representing the value of the variable after traveling 
along the edge. 
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As usual, we define a state of a hybrid system as a pair {L, (p) where L € Cis 
a location and (p: X M is a valuation of the data variables, (p naturally extends 
to (constraint) terms and (constraint) formulas. A state (L, (p) is called admissible 
if <p(inv(L)) holds. Given two admissible states s = (L,p) and s' = (L',cp') we 
say that s' is transition-reachable from s - denoted by s s' - if there exists 
a transition t = {L, L') G £ with source L and target L' , and both (p{guard{t)) 
and 4>'{x) = 4>{act{t,x)) for each x G X. We call s' timely-reachable from s with 
delay 6 - denoted by s s' - where (5 is a non-negative real number, ii L = L' 
and for each x G X there exists a differentiable function f^-. [0,i5] i— > M, with 
the first derivative fx'- (0><5) M, such that (1) fx(Q) = (p{x) and fx{5) = (p' {x) 
and (2) for all e G M with 0 < e < i5: both mv{L)[x\/ fx^{e ), . . . ^Xn/ and 
fx{^) = dif{L,x)[xi/ fxi{e), ■ ■ ■ ,Xn/ fx„{f)] are true, s' is timely-reachable from 
s - denoted by s i-b s' - if there exists a non- negative i5 G M such that s s', 
s' is said to be reachable from s if (s, s') G (i^ U A)*. A run p oiTi with initial 
state (Jo = (Lqj 0 o) is a maximal sequence of states represented as 



P = fTo (71 (72 (73 ■ ■ ■ 

where ti G and /^ : [0,ti] i-^- (A i— > M), such that (i) /i(0) = (pi, (ii) 
inv(Lj)[A//j(t)(A)] holds for all 0 < t < U, (iii) fipti)) (Ji+i and (iv) 
for all 0 < t' < t' -I- i5 < ti : {Li, fi{t')) (i- {Li, fi{t' -\- 6)). The set of states con- 
tained in such a run p is given as {{Li, fi{t)) | t G M, 0 < t < ti}. The set of all 
runs of a hybrid system Ti. with initial state cr is denoted by runs(7t, a). A posi- 
tion 7T of a run p = (Tq ai (73 ■ is a pair tt = (t, r) G N x M 

such that 0 < r < ti. Positions are ordered lexicographically, i.e., (z,r) < (j, s) 
if and only if z < j or (z = j and r < s). Also, {i,r) < {j,s) if and only if 
(z,r) < {j,s) or (z = j and r = s). By p{tt) with tt = (z,r) we denote the state 
{Li, Mr)). 

A run is said to be non-zeno if ^ ti diverges. In the sequel we assume that the 
runs of the hybrid system under consideration are all non-zeno.^ 

In order to formulate properties of hybrid systems we consider (a fragment 
of) ICTL, the Integrator Computation Tree Logic [AHH96]. For simplicity we 
omit the Until-operators in this paper. Their introduction does not cause much 
more effort, though (see [Non99]). 

Given some hybrid system with locations C and data variables X, the set of 
ICTL formulas is defined as the smallest set containing all constraint formulas 
from CF over X, all location names from C, and that is closed under the usual 
boolean connectives together with temporal (ICTL) operators AG , EF , EG , 
and AF . Moreover, if <P is an ICTL formula, z is a new data variable, and 
[Li, . . . , Ln} G C then L-n} (p is an ICTL formula as well. Intuitively, the 

^ The assumption of non-zenoness implies that hybrid systems are deadlock-free, i.e., 
there is no reachable state that has no successor. So-called livelocks, however, are not 
excluded. This means that we absolutely allow states which have only themselves as 
future states. The latter case just states that the situation does not change in time, 
whereas the former case (deadlock) would claim that time itself has come to an end. 
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temporal operators AG AF <P, EG EF ^ mean “always” , “inevitably” , “pos- 
sibly always” , and “possibly” respectively. Their formal semantics with respect 
to hybrid systems is defined below. 



Definition 1. Given a hybrid system H = {X,£,£,dif,inv, guard, act) and a 
state a = {L,(j)), the semantics of ICTL with respect to H and a is defined as: 



H,cr \= c 
n,a^N 
7i,a \= 

Tt,a \= 'P AF 

n,a^ AGP 
H,cr\=EGP 
Ti.,a \= .P 



iff 1= 4>{c) for constraint formula c 
iff locations N and L are identical 

iff H,a^P 

iff H,cr \= P H,a F 

and similarly for the other boolean connectives 
iff '^P {p € runs{H,a) Vtt (tt G positions{p) ^ \= P)) 

iff 3p {p G runs{H, a) & Vtt (tt G positions(p) H, p{tt) ^ P)) 
iff ,{L,4>[z/0]) \= P, where If G C, 



where EE P = ^AG ^P and AF P = ^EG ^P. By we mean the 

extended system we obtain by adding the new clock z (initialized with 0) which is 
supposed to run with slope 1 within locations L\, . . . ,Ln and with slope 0, i.e., 
it is stopped, for all other locations. 

Given a variable valuation (j> we define the new valuation </'[z/0] as 



(p[z/0]{x) 



(p{x) ifx^z 
0 otherwise. 



3 The Verification Approach 

Here we restrict our view to linear hybrid systems, where difiL, x) is a constant, 
say kf, for each location L and data variable x. This restriction can easily be 
weakened to rectangular hybrid systems (where difiL, x) is given as an interval 
of reals) without any real effort. For a better readability we denote sequences of 
the form x\ + kffS, ... ,Xn + kff"6 by AT -|- k^6 where X = {x\, . . . , x„}, and, 
similarly we mean L{act{T, x\),. . . , act{T, x„)) whenever we write L{act{T, AT)). 

3.1 First-Order Theories for Reachability and Inevitability 

As usual, an interpretation 9 = (D,'Ac,(l>) for a first-order theory associated 
with a hybrid system H with locations C has a fixed domain T> (the reals or 
the rationals, say), a valuation (j) for the data variables in X, and a meaning 
function for the locations in C such that 9c (T) G H”, where n is the number 
of data variables in A. A model of a formula P is an interpretation satisfying 
this formula. 

We often also speak of a model as a set of ground atoms of the form 

{L(9(ti), . . . , 9(t„)) I 9 \= L{ti , . . . , tn)}, where ti are constraint terms} 
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where 9 is a model in the above sense. Interpretations (models) are partially 
ordered by set-inclusion. A minimal model of ^ is a model of such that there 
exists no proper subset of it that also satisfies 

We now define two different kind of first-order theories for a given hybrid 
system: one that is responsible for the possible states, and one that is responsible 
for the unavoidable states. 

Definition 2 (Reachability Theory). Let Ti = {X, £,£, dif, inv, guard, act) 
be a hybrid system. For each L € C we define the first-order theory 



VA L{X) 



inv{L) A 

V(5 ((5 > 0 A inv{L)[X/X -b k^6] L{X + k^6)) A 
y/y guard{T) ^ N {act{T , X)) 

T=(L,N)&e 



as the local reachability theory of L in Ti, VJfi for short. By the reachability 
theory of - which we call TZ-h, or simply TZ if Ti is clear from the context - we 
understand the conjunction of all local reachability theories, i.e., TZ-h — 

Lee 

What the reachability theory expresses is that (for each location L) (i) the 
location invariant must hold, (ii) that there is a possible time transition, and 
(iii) for each outgoing edge: if the enabling guard is true then the target location 
can be reached provided the corresponding discrete action has been peformed. 

Definitions (Inevitability Theory). Let Ti = (X, £,£, dif, inv, guard, act) 
be a hybrid system. For each L G £ we define the first-order theory 



VA L(A) 



VA L(A) 



inv{L) 

{y6 5>Q^ L{X + k^5) V 
r (5 > 0 A 

\/5' t)<6' <6^ L{X + k^5') A 



35 



V 

T=(L,N)e£ 



guard(r)[A/A -b k^6] A 

N{act{T,X)[X/X + k^S\) 



as the local inevitability theory of L in Ti, for short. By the inevitability 
theory oiTi - which we call F-h, or simply 2 if Ti is clear from the context - we 
understand the conjunction of all local inevitability theories, i.e.. In = A 

Lee 

The inevitability theory might require some more explanation. In a sense it ex- 
presses (for each given state) between which possibilities the system can choose. 
The first part of any local inevitability theory is trivial. It just guarantees the 
mere fact that for each location predicate the corresponding location invariant 
is supposed to hold. The second part is more complicated and more interesting. 
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Note that, given an arbitrary state represented by the location predicate L{X), 
either the system remains forever in this location, i.e., V(5 i5 > 0 — > L{X + k^6), 
or it will sooner or later leave this very location. In the latter case we know that 
there is a time delay S after which one of the guards of the outgoing edges is 
true and until then the system remains within location L. This is exactly what 
is expressed by the complicated second part of the local inevitability theories. 

Intuitively, the reachability theory tells us what can be done in certain sit- 
uations (states), whereas the inevitability theory describes what must be done, 
it collects all the immediate future possibilities. 

The importance of these two theories will become apparent from the following 
Lemma. 

Lemma 1. Given a hybrid system H and an initial state {L,(j)). 

— The (unique) minimal model of L{(j){X)) /\TZ -h corresponds to the set of states 
that are reachable from (L, f) in the hybrid system TL. 

— Each minimal model of L{(f>{X)) /\Tu corresponds to the set of states of one 
of the runs of TL. 

— The set of states of each run ofTL forms a model of L{<f>{X)) TTu- 
Proof. Can be found in [Non99]. 

The above lemma provides us with a formal connection between the reachability 
theory (inevitability theory) and the reachable (inevitable) states. Briefly, <P 
holds always (AG <P) iff <P holds in all reachable states iff <P holds for every 
element in the unique minimal model of the reachability theory (together with 
the initial state) iff <P holds for every element of some model of the reachability 
theory (together with the initial state). This observation leads to the following 
definition and main theorem. 

Definition 4 (Characteristic Constraint Formulas). The (second-order) 
formula associated with an ICTL formula <P, the hybrid system TL, and location 
L, , representing a characteristic constraint formula for given TL in L, 

is recursively defined by 

=c 

rj-nL(x) _ J T if L and L' are identical 
^ (T otherwise 

and similarly for the other boolean connectives 
\z^ where Af C C 

=3Li,... ,L„L(A^)A7^HAAwe£ VX N{X) ^ 

=3Li,... ,L„L(X)AXhAA^6£ yX N{X) ^ 
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Theorem 1. Given a hybrid system H with data variables X , an initial state 
{L, (j>) and an ICTL formula <P. Then 

iff 

Proof. By induction on the structure of<P. 

The base cases are trivial. Also in case of a boolean connective there are no 
problems at all. The induction steps are exemplified for the case = AG T. For 
the other cases see [Non99]. 

AG F 

iff TC,a \=F for every a reachable from {L, <f>) 

iff Vcr ((L, (f), (t) e (A u A)* ^H,a[=T' 

iff WN, 4>' N{<f'{X)) e minMod{L{(f){X)) A Tin) =A H, {N, (f) ^ T 
(Lemma 1 ) 

zjff 39 9 h A & V7V, {N, (/)') G 9 ^ H, {N, (f) ^ T 

39 9 h L{f{X)) ATZnk V7V, </.' (TV, <(,') G 9 ^ h 

(induction hypothesis) 

zjOP 39 9 h L{f{X)) A 7^H & 9 h VX N{X) ^ 
zjOP 39 9 h H<KX)) A Tin A VAT N{X) -> 
iff h3Li,... ,L„L(<^(X))A7^HAAive£V^^W-^ 

iff 



3.2 Eliminating Locations 



Theorem 1 tells us that we can solve a hybrid system verification problem by 
proving the satisfiability of some suitable first-order theory, or equivalently, by 
showing the validity of some corresponding second-order formula. The Elimi- 
nation Theorem below helps us in this respect, for it allows us to transform a 
given second-order formula into an equivalent first-order formula (if this is at all 
possible).^ 



^ In general, second-order formulas do not necessarily have a (finite or infinite) first- 
order equivalent. However, in case the second-order formula is of the form 3P F, 
where F is a. first-order formula that is Horn in P, then we know that there exists 
a first-order equivalent (which may be infinite, though). Note that proving Safety 
properties, i.e. proving the validity of second-order formulas involving reachability 
theories, are just of this form. 
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Theorem 2 (Elimination Theorem). Let<P andW be two first-order formulas 
that are positive with respect to the predicate symbol L. Then 

'm/{vL{x).T>y_ 

where vL{x).‘T{L) = f\<P\T) with <l>0(T) = T, ^>"+i(T) = ^>(^”(T)) 

2<o; 



3L (Vx (L(x) ^ at) = T 



The proof of this Theorem can be found in [NS95] (but also see [NS99] and 
[NOS99]). There also some generalizations and dual forms are examined. For 
the purpose of this paper, however, the above form suffices. 

The Elimination Theorem tells us that any second-order formula of the form 
3L (Vx {L{x) ^ <P) at) is equivalent to T with every occurrence of L (with 
actual argument list a) within T replaced by the greatest fixpoint of T (after 
instantiating the abstract parameters with the actual arguments). Notice that 
the second-order formulas we are dealing with are indeed of the form required. 
Therefore, with each application of the Elimination Theorem we get rid of one of 
the existentially quantified predicate symbols. Now, since these predicate sym- 
bols are just the location names of the hybrid system under consideration, each 
application of the Elimination Theorem also eliminates one of the locations. 

Evidently, it cannot be guaranteed that the fixpoint computation will termi- 
nate in general. However, it can easily be shown (see [Non99]) that in case we 
are about to eliminate a location which has no outgoing edge leading to itself, 
the fixpoint computation will definitely terminate after two iterations. 

Coming back to the example on page 353 where we wanted to examine the 
effect of eliminating location L^, we now know that we have to compute - in 
fact, find a first-order equivalent for - the second-order formula 





r Va;, y Li{x,y) ^ X < y ^ L2{x, y) A 






X <y A 1 


3L2 


\/x,y L2{x,y) ^ < 


3 ; + y < 10 A [ 

Vi 5 (( 5 >OAa;-l- 2 ( 5 < 2 / + i5— > ^ 2(3 -\- 25 ,y -\- 6)) A ( 






a; = y^ 1 . 3 ( 0 , 0 ) J 



The five conjuncts of the above second-order formula describe the transition 
from L\ to L 2 , the location invariant for L 2 , the property to be proved, the time 
transition for location L 2 , and the edge transition from L 2 to L 3 respectively. 
In order to apply the Elimination Theorem to this second-order formula, let 
T = Vx, y Li{x,y) ^ x < y — > L 2 {x, y) and 'T = x<yAx-\-y< lOAVJ (i5> 
OAa;-k2(5<?/-h(5^ ^ 2 ( 3 ; + 26,y + 5)) A x = y ^ ^ 3 ( 0 , 0). Then <I>°(T) = T, 
^^(T) = a: < y A a; -I- y < 10 A a: = 2 / ^ ^ 3 ( 0 , 0), ^^(T) = ^^(T) A a: < 2 / — > 2y < 
5 -I- a:, and <?^(T) = <?^(T) as can easily be checked by the reader. We thus have 
found the fixpoint and substitute it for L 2 in T resulting in 

Va:, y Li{x,y) ^ x <y ^ 0) 

Va;, y Li{x,y) ^ X < y ^ 2y < X 5 

The first formula describes just the new edge to be introduced. The second 
formula, on the other hand, tells us about the necessary and sufficient condition 
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on the data variables for location Li such that it would be impossible to violate 
+ y < 10 in location L 2 - 



3.3 Examples and Experimental Results 



There exists a prototype implementation of the Elimination Approach (for prov- 
ing safety-properties) written in Sicstus-Prolog with the CLP(Q,R)-library for 
constraint handling. It has been tested on a lot of examples known from the 
literature (or taken from hybrid system verifier distributions). The experimental 
results can briefly be summarized as follows: as already expected in the introduc- 
tion, standard forward reachability (if it at all terminates) can hardly be beaten 
in case we are about to prove safety properties for non-trivial systems that only 
require a single pass through the reachable locations. This is the case for in- 
stance for the famous “audio-protocoF’-example.^ For other, unfortunately still 
trivial systems like the “Leaking Gas Burner” or the “Billiards” -example, the 
Elimination Approach showed a slightly better behavior than standard reacha- 
bility analysis.^ However, in such cases, where safety properties can be proved in 
milliseconds anyway, this can hardly be called evidence. The lack of non-trivial 
hybrid system in the literature that require several passes through some of their 
locations made us compose our own examples. They are designed as simple as 
possible such that they may serve to illustrate the effect of the Elimination Ap- 
proach compared to reachability analysis methods. Two such examples are given 
below. 



A Silly Multiplier. This is an example within which three numbers a, b, and 
c are to be multiplied and the final product is stored in the data variable p. The 
multiplication is performed by successively adding 1 to p, similar to the nested 
for-loop 



for {w = 0;w < c; w-l— k) 
for {v = 0]v < b; u-k- k) 
for {u = 0;u < a; u-k-k) 

p := p + 1 



® In case of more trivial such examples like the “Water-Level-Monitor” or the 
“Railroad-Gate-Controller” there is not so much of a difference. 

^ It should be noted here that it is in fact very easy to compare the Elimination 
Approach with standard reachability analysis methods for, in a sense, reachability 
analysis can be viewed as a special case of the Elimination Approach: we just have 
to move the location names to the argument list as a further additional argument 
(leaving a single unique dummy predicate as the only remaining predicate symbol) . 
This then leads to backward reachability, whereas, by using a dual form of the 
Elimination Theorem, we can also get forward reachability (see also [Non99]). 
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a; = 1 




It is to be shown that the location F can be reached - after all, as soon as F is 
reached, the data variable p contains the multiplication result we are interested 
in. (Backward or forward) reachability analysis in a sense simulates the behavior 
of the multiplier. I.e., since this system is fully deterministic, it takes a walk 
through the whole computation. Evidently, this is very time consuming even 
if we only attempted to compute 10 x 10 x 10. Now, compare this with the 
Elimination Approach. It takes two steps to eliminate the top left location, 
another 2a iterations for the bottom left location, some 2b iterations for the 
bottom right location, and finally, 2c iterations for the top right location. We 
then end up with a single location, namely F, for which the data variable p is 
initialized with the product ax b x c. Thus, even for big numbers a, b, and c the 
elimination approach is able to fulfill its task in a very short amount of time, 
whereas any kind of reachability analysis requires much more effort (essentially 
a + b+ c versus axb x c). 

An Example Where Reachability Fails. The particularity about the fol- 
lowing example is that it contains an “impossible” transition, i.e., one of the 
locations - the one at the bottom - is unreachable because the guard {y = 2) 
of the transition that may lead to this very location will never be enabled. In 
a sense, forward reachability analysis detects this, though rather indirectly, for 
it never tries to compute states involving this location. However, forward reach- 
ability nevertheless does not terminate, since the data variable y may become 
arbitrarily big and therefore the fixpoint computation finds new possible states 
in each iteration. At the first glance, backward reachability might have a better 
chance. Suppose we were about to prove that a; < y is an overall invariant of the 
system. If there were not the bottom location, backward reachability would have 
no problem to detect that the invariant indeed holds. But it this very location (or 
actually the transition that leads to itself) that prevents backward reachability 
from termination. 
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As for the Elimination Approach, recall that the real purpose of the Elimina- 
tion Theorem is to provide us with a model for the given first-order theory. 
The Elimination Theorem is just one of the possible means we can imagine that 
may help us finding such a model. There are, however, cases for which we can 
(even by mere syntactic examinations) find out that a given first-order formula 
indeed has a (trivial) model. This is the case, for instance, if we reach a situation 
(maybe after having performed some eliminations) where every conjunct of the 
intermediate result contains a negative location predicate literal. In this case we 
can conclude that all locations that have not yet been eliminated are unreach- 
able. This is in fact very common. The famous “Railroad-Gate-Controller” for 
example, has 22 (compound) locations. But only 7 of them are in fact reachable. 
Our implementation of the Elimination Approach first eliminates these seven 
reachable locations and then detects that all others are unreachable and ter- 
minates with success. The same happens for the example above: first the top 
three locations get eliminated and then the unreachability of the final (bottom) 
location is detected. Thus the system terminates with success. 

4 Summary and Conclusion 

We presented a hybrid systems verification methodology that is based on a suc- 
cessive location elimination procedure that allows us to simplify the system until 
it becomes trivial. This approach turns out to be particularly useful in case stan- 
dard forward or backward reachability analysis methods would require to pass 
some of the locations several times (long lasting loops). There are even exam- 
ples where a reachability analysis fails, whereas the Elimination Approach ter- 
minates successfully. However, there also are some non-trivial examples where 
forward reachability seems unbeatable, namely systems which contain quite a 
huge amount of locations, yet require merely a single pass in order to compute 
the set of reachable states. This observation leads to the conjecture that it might 
make sense to suitably combine the two methodologies: in case a non-trivial sys- 
tem (with many locations) contains long-lasting loops that prevent a reachability 
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analysis from termination within some reasonable amount of time, it certainly 
makes sense to try and eliminate such loops with the help of the Elimination 
Approach and maybe to proceed with the reachability analysis after that. 
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Abstract. Switching linear dynamic systems (SLDS) attempt to de- 
scribe a complex nonlinear dynamic system with a succession of linear 
models indexed by a switching variable. Unfortunately, despite SLDS’s 
simplicity exact state and parameter estimation are still intractable. Re- 
cently, a broad class of learning and inference algorithms for time-series 
models have been successfully cast in the framework of dynamic Bayesian 
networks (DBNs). This paper describes a novel DBN-based SLDS model. 
A key feature of our approach are two approximate inference techniques 
for overcoming the intractability of exact inference in SLDS. As an exam- 
ple, we apply our model to the human figure motion analysis. We present 
experimental results for learning figure dynamics from video data and 
show promising results for tracking, interpolation, synthesis, and classi- 
fication using learned models. 



1 Introduction 

Many natural processes have complex, highly nonlinear and time-varying dy- 
namics. For instance, economic trends, maneuvering targets, and the human 
figure all exhibit complex and rich dynamic behavior. Dynamics are essential to 
the analysis of these processes as well as to their realistic prediction (forecasting) 
and synthesis (simulation). Dynamic models can provide a powerful cue in the 
presence of missing/multiple measurements and measurement noise. A dynamic 
model imposes additional structure on the state space by specifying which state 
trajectories are possible (or probable) and by specifying the speed at which a 
trajectory evolves. 

Unfortunately, state and parameter estimation problems in complex dynamic 
models can be a daunting task. State estimation in non-linear models is usually 
cast in frameworks whose origins lay in the theory of extended Kalman filters 
(c.f. [1]). Parameter estimation of such highly nonlinear models is often a result 
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of tedious measurements and expert knowledge about the problem. For instance, 
consider the human figure modeling in the field of biomechanics. The dynamics 
of the figure are the result of its mass distribution, joint torques produced by 
the motor control system, and reaction forces resulting from contact with the 
environment (e.g. the floor). Research efforts in biomechanics, rehabilitation, and 
sports medicine have resulted in complex, specialized models of human motion 
(c.f. [11].) Such complex models have been used successfully to simulate [10] and 
to track human body motion [27]. 

This paper explores the alternative method of learning dynamic models from 
a training corpus of observed state space trajectories. In cases where sufficient 
training data is available, the learning approach promises flexibility and general- 
ity. A wide range of learning algorithms can be cast in the framework of dynamic 
Bayesian networks (DBNs) [7], a subclass of now famous Bayesian network mod- 
els (c.f. [23, 13]). DBNs generalize two well-known signal modeling tools: Kalman 
filters [1] for continuous state linear dynamic systems (LDS) and Hidden Markov 
Models (HMMs) [24] for classification of discrete state sequences. 

The DBN framework provides two distinct benefits: First, a broad vari- 
ety of modeling schemes can be conceptualized in a single framework with an 
intuitively-appealing graphical notation (see Figure 1 for an example). Second, 
a broad corpus of exact and approximate statistical inference and learning tech- 
niques from the Bayesian network literature can be applied to dynamical sys- 
tems. In particular, it has been shown that estimation in LDSs and inference in 
HMMs are special cases of inference in DBNs. 

The focus of this paper is on a subclass of DBN models called Switching 
Linear Dynamic Systems [2, 26, 17, 9, 22]. Intuitively, these models attempt to 
describe a complex nonlinear dynamic system with a succession of linear models 
that are indexed by a switching variable. While other approaches such as learning 
weighted combinations of linear models are possible, the switching approach has 
an appealing simplicity and is naturally suited to the case where the dynamics 
are time-varying. 

This paper makes two contributions. First, we derive two efficient algorithms 
for approximate state estimation in SLDSs. An approximate Viterbi inference 
algorithm and a structured variational inference algorithm are cast as in the 
framework of DBN inference. Second, we demonstrate the application of the 
SLDS framework to modeling the human figure dynamics. In particular, we 
demonstrate the learning of switching models of fronto-parallel walking and jog- 
ging motion from video data. We demonstrate the application of these learned 
models to segmentation, synthesis, and tracking tasks. 



2 Switching Linear Dynamic System Model 

Consider an SLDS described using the following set of continuous and discrete 
state-space equations: 



xt+i = A{st+i)xt + vt+i{st+i), yt = Cxt + wt 
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for the continuous- valued linear dynamic system (LDS), and 

Pr{st+i\st) = St 

for the discrete switching model. We assumed that the LDS models a Gauss- 
Markov process with state noise Vt{st) ~ Af(0, Q(st)), measurement noise Wt ~ 
Af{0,R), and initial state xq ~ A/’(a;o(so), Qo(so))- The switching model is as- 
sumed to be a discrete first order Markov process. State variables of this model 
are written as St- They belong to the set of S discrete symbols {cq, . . . , es_i}, 
where et is the unit vector of dimension S with a non-zero element in the i-th po- 
sition. The switching model is defined with the state transition matrix U whose 
elements are = Pr(st+i = €i\st = Cj), and an initial state distribution 

TTq. 

Coupling between the LDS and the switching process stems from the de- 
pendency of the LDS parameters A and Q on the switching process state St- 
Namely, 



A(^St — Ci) — Aiy Qi^St — Cj) — Qi 

In other words, switching state St determines which of S possible plant models 
is used at time t. 

The complex state space representation is equivalently depicted by the DBN 
dependency graph in Figure 1. The dependency graph implies that the joint 




Fig. 1. Bayesian network representation (dependency graph) of an SLDS of duration 
five, s denote instances of the discrete valued switching states, x and y are continuous 
valued LDS states and measurements. Arcs in the graph show dependencies among 
variables. 



distribution P over the variables of the SLDS can be written as 
P{yr, Tt, St) = 

Pr{so) Y\^~^ Pr(st|st_i)Pr(a:o|so) YlJji Pr{xt\xt-i, St) Pr{yt\xt), 

where yx, Xt, and St denote the sequences (of length T) of observations and hid- 
den state variables. For instance, = {yo, ■ ■ ■ , yr-i}- From the Gauss-Markov 
assumption on the LDS (e.g. Xt+i\xt,st+i=ei ~ Af{AiXt,Qi)) and recalling the 
Markov switching model assumption, the joint pdf of the SLDS of duration T 
can be easily defined. 
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2.1 Hidden State Inference and Estimation 

The goal of inference in complex DBNs is to estimate the posterior probability 
of the hidden states of the system {st and Xt) given some known sequence of 
observations and the known model parameters. Namely, we need to find the 
posterior 

= Pr{XT,ST\yT), 

or , equivalently, its sufficient statistics. Given the form of P it is easy to show 
that these are the first and the second order statistics: mean and covariance 
among hidden states Xt,Xt-i, St, St-i- 

If there were no switching dynamics, the inference would be straightforward 
- we could infer Xt from using LDS inference (RTS smoothing [25]). How- 
ever, the presence of switching dynamics embedded in matrix U makes exact 
inference more complicated. To see that, assume that the initial distribution of 
xo at t = 0 is Gaussian, at t = 1 the pdf of the physical system state xi becomes 
a mixture of S Gaussian pdfs since we need to marginalize over S possible but 
unknown plant models. At time t we will have a mixture of S* Gaussians, which 
is clearly intractable for even moderate sequence lengths. It is therefore neces- 
sary to explore approximate inference techniques that will result in a tractable 
learning method. 

2.2 Approximate Inference Using Viterbi Approximation 

The task of Viterbi approximation approach is to find the most likely sequence 
of switching states St for a given observation sequence Vt- If the best sequence 
of switching states is denoted we can then approximate the desired posterior 
R(At,5t|Vt) as^ 

R(At,5t|Vt) = R(Ar|5r, Vt)R(5t|Vt) ~ R(At|5t,Vt) S{St — S^), 

i.e. the switching sequence posterior P(5 t|Vt) was approximated by its mode. 
It is well known how to apply Viterbi inference to discrete state hidden Markov 
models [24] and continuous state Gauss-Markov models [16]. Here we develop an 
algorithm for approximate Viterbi inference in SLDSs. 

More formally, we are looking for the switching sequence such that 

St = argmaxP(5T|VT)- 

St 

It is easily to shown that a (suboptimal) solution to this problem can be obtain 
by recursive optimization of the probability of the best sequence at time t 

Jt,r = maxP(5t_i,St = ei,Vt) 

St-1 

« max{p (yt\st = e^St-i = e^-, 5(*_2(j), Vi-i) P {st = ei\st-i = Cj) 
ma,xP {St- 2 , st-i = ej,yt-i)\ (1) 

St~2 J 



^ 5{x) = 1 for X = % and zero otherwise. 
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Here S^_ 2 {i) is the “best” switching sequence up to time t — 1 when SLDS is in 
state i at time t — 1, S^_ 2 (i) = argmax 5 j _2 

To find the likelihood term P{yt) in Equation 1 note that concurrently with 
the recursion for each pair of consecutive switching state i,j at times t,t—l one 
can update the statistics ^ of continuous states of LDS based on current “best” 
switching sequences using the Kalman filter inference (c.f. [1]). For instance, 

xt\t,i = {xt\yt,st = 

~ — — Cj, = Cj , (j)) 

“ (^Xt\yt^ St = St — \ = , S'i_2(j)) ■ 

denote corresponding second order statistics. The likelihood 
term can then be easily computed as the probability of innovation yt — Cxt\t-i,i,j 
oi j ^ i transition, which has normal distribution with mean Cxt\t-i,i,j smd 
variance C T P- 

The index of the most likely state j* at t — 1 corresponding to the maximum 
in Equation 1 is kept for every state i at time t in the state transition record 
= j*- LDS statistics corresponding to j* are updated accordingly, Xt\t,i = 
Xt\t, i, tpt-i i- Once all T observations have been fused to the 

“best” switching state sequence is the one that ends in = argmin^ 

States of this sequence can be traced back through the state transition record 
l,i) b = 

Once the “best” switching sequence is known, the switching model’s sufficient 
statistics are simply (st) = Cj* and = ei*e'. Sufficient LDS statistics 

for this switching sequence can be easily obtained using Rauch-Tung-Streiber 
(RTS) fixed interval smoothing [1]. 

The Viterbi inference algorithm for SLDSs can now be summarized as 



Find most likely state sequence using recursion in 1; 
Find DBN’s sufficient statistics for P(5 t|3^t) = S{St — 



Approximate Inference Using Structured Variational Inference Gen- 
eral structured variational inference technique for Bayesian networks is described 
in [14]. The idea behind this method is to find distribution Q which is in some 
sense close to the desired conditional distribution P, but is easier to com- 
pute. One can then employ Q as an approximation of P, P(Ar,5T|3fT) ~ 
Q{Xt, 5t|Vt)- 

Namely, for a given set of observations Vt, a distribution Q(ftT,5T|?7,3fT) 
with an additional set of variational parameters rj is defined such that Kullback- 
Leibler divergence between Q(Ar, 5 t|?7, Vt) and P{XT,ST\yT) is minimized 



^ LDS statistics are state means and covariances. (.) denotes the expectation operator. 
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Fig. 2. Factorization of the original SLDS. Factorization reduces the fully cou- 
pled model into a seemingly decoupled pair of a HMM (Qs) and a LDS (Qx)- 



with respect to rj: 



V 



* 




Q{Xt, |?7, yx) log 



P{Xt, tST|yr) 
Q{XT,ST\r]^yT) 



The dependency structure of Q is chosen such that it closely resembles the depen- 
dency structure of the original distribution P. However, unlike P the dependency 
structure of Q must allow a computationally efficient inference. In our case we 
define Q by decoupling the switching and LDS portions of SLDS as shown in Fig- 
ure 2. The two subgraphs of the original network are a hidden Markov model 
(HMM) Qs with variational parameters {qo, . . . , qx-i} and a time-varying LDS 
Qx with variational parameters {aio, Hq, . . . , At-i, Qo, - ■ ■ , Qt-i\- Because the 
subgraphs are decoupled, inference can be performed for each submodel sep- 
arately, Q{XT,ST\rj,yT) = Qjc(Tt|j7, 3^t) Qs(5t|j 7). This is also reflected in 
the sufficient statistics of the posterior defined by the approximating network, 
e.g. (xtxt'st) = {xtxt) (st). 

The optimal values of the variational parameters ry are obtained by minimiz- 
ing the KL-divergence w.r.t. tj. This leads to, for instance, the following recursive 
expression for the LDS Qx’s state transition matrix 



s-i 

At = QtY,Q-^A{st{A). ( 2 ) 

Similarly, an expression can be found for optimal HMM variational parameters 
loggt(z) = -i (^{xt - A^xt-i)' Q~^ {xt - A,xt-i)'j - ^ log \Qt,i\- (3) 

To obtain the expectation terms (st) = Pr{st\qo, - ■ - ,qT-i) we use the in- 
ference in the HMM with output “probabilities” qt [24]. Similarly, to obtain 
(xt) = E[xt\yT] we perform LDS inference in the decoupled time- varying LDS 
via RTS smoothing. Since At,Qt in the decoupled LDS Qx depends on (st) 
from the decoupled HMM Qs and qt depends on (xt) , (xtXt) , (xtXt-i) from 
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the decoupled LDS, the optimal parameter equations (e.g. 2 and 3) together 
with the inference solutions in the decoupled models form a set of fixed-point 
equations. Solution of this fixed-point set yields a tractable approximation to 
the intractable inference of the original fully coupled SLDS. 

The variational inference algorithm for fully coupled SLDSs can now be sum- 
marized as: 



error = oo; 

Initialize (st); 

while (error > maxError) { 

Find Qt,A,xo from (st); 

Estimate (xt) , (xtXt) and {xtXt-i) from yt 
using time-varying LDS inference; 

Find qt from (xt ) , (xtXt) and {xtXt-i}', 
Estimate (st) from qt using HMM inference. 
Update approximation error (KL divergence); 



2.3 Maximum Likelihood Learning of Complex DBNs 

Learning in complex DBNs can be formulated as the problem of ML learning in 
general Bayesian networks. Hence, a generalized EM algorithm [21] can be used 
to find optimal values of DBN parameters {H, C, Q, R, II, tto}. The expectation 
(E) step of EM is the inference itself. We outlined two approximate inference 
algorithms in the previous section. Note that the variational inference algorithm 
does not necessarily lead to non-decreasing likelihood of data in the EM (even 
though it usually does so in practice.) On the other hand, (a bound on) likeli- 
hood is guaranteed not to decrease when structural variational inference is used. 
See [14] for more details. 

Given the sufficient statistics from the inference phase, parameter update 
equations in maximization (M) step are obtained by maximizing {log P{Xt, St, 
yx) with respect to the parameter of interest. For instance, updated values of 
the state transition parameters are easily shown to be 



Ai — 

fl = 



/T-l \ /T-1 \ 1 

^ {xtx't_^St{i)) I I ^ {xt-ix't_^St{i)) I 
Vt=i / \t=i / 



/T-l \ /T-1 

^ diag ^ 



(Si) 



All the variable statistics are evaluated before updating any parameters. Notice 
that the above equations represent a generalization of the parameter update 
equations of classical (non-switching) LDS models [8]. 
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3 Previous Work 

SLDS models and their equivalents have been studied in statistics, time-series 
modeling, and target tracking since early 1970’s. Bar-Shalom [2] and Kim [17] 
have developed a number of approximate pseudo-Bayesian inference techniques 
based on mixture component truncation or collapsing is SLDSs. They did not 
address the issue of learning system parameters. Shumway and Stoffer [26] pre- 
sented a systematic view of inference and learning in SLDS while assuming 
known prior switching state distributions at each time instance, Pr{st) = 
and no temporal dependency between switching states. Krishnamurthy and 
Evans [18] imposed Markov dynamics on the switching model. However, they 
assumed that noisy measurements of the switching states are available. 

Ghahramani [9] introduced a DBN-framework for learning and approximate 
inference in one class of SLDS models. His underlying model differs from ours in 
assuming the presence of S independent, white noise-driven LDSs whose mea- 
surements are selected by the Markov switching process. An alternative input- 
switching LDS model was proposed by Pavlovic et al. [22] and utilized for mouse 
motion classification. A switching model framework for particle filters is de- 
scribed in [12] and applied to dynamics learning in [3]. Manifold learning [5] is 
another approach to constraining the set of allowable trajectories within a high 
dimensional state space. An HMM-based approach is described in [4]. 

4 Experiments 

We applied our DBN-based SLDS framework to the modeling of motion of the 
human figure. Most current models of the human figure dynamics belong to one 
of two model groups. One assumes highly complex, hand-crafted biomechanical 
models. This approach has been used successfully to produce computer graphics 
animations of human motion [10] and to track upper body motion in a user- 
interface setting [27]. On the other end of the spectrum are simple LDS models. 
Most previous figure trackers which have used a dynamic model employed a 
simple smoothness prior such as a constant velocity Kalman filter [15]. 

Two categories of fronto-parallel motion were present in our data: walking 
and jogging. Fronto-parallel motions exhibit interesting dynamics and are free 
from the difficulties of 3-D reconstruction. Experiments can be conducted easily 
using a single video source, while self-occlusions and cluttered backgrounds make 
the tracking problem non-trivial. 

We adopted the 2-D Scaled Prismatic Model proposed by Morris and 
Rehg [19] to describe the kinematics of the figure. The kinematic model lies 
in the image plane, with each link having one degree of freedom (DOF) in ro- 
tation and another DOF in length. A chain of SPM transforms can model the 
image displacement and foreshortening effects produced by 3-D rigid links. The 
appearance of each link in the image is described by a template of pixels which 
is manually initialized and deformed by the link’s DOF’s. 
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In our figure tracking experiments we analyzed the motion of the legs, torso, 
and head, and ignoring the arms. Our kinematic model had eight DOF’s, cor- 
responding to rotations at the knees, hip, and neck. A sample configuration of 
our figure model is shown in Figure 4.2. 



4.1 Classification 

The first task we addressed was learning an SLDS model for walking and run- 
ning. Our training set consisted of 18 sequences of six individuals jogging (two 
examples of three people) and walking at a moderate pace (two examples of 
six people.) Each sequence was approximately 50 frames duration. The training 
data consisted of the joint angle states of the SPM in each image frame, which 
was obtained manually. 

Each of the two motion types were each modeled as multi-state^ SLDSs 
and then combined into a single complex SLDS. Measurement matrix in all 
cases was assumed to be identity, C = I. Initial state segmentation within each 
motion type was obtained using unsupervised clustering in a state space of some 
simple dynamics model (e.g. constant velocity model.) Parameters of the model 
(A, Q, i?, Xq, fl, ttq) were then reestimated using the EM-learning framework with 
approximate Viterbi inference. This yielded refined segmentation of switching 
states within each of the models. An example of the learned switching state 
sequence within a single “jog” training example is shown in Figure 3(a). 




(a) (b) 

Fig. 3. (a) Segmentation of two-state SLDS model states within single “jog” mo- 
tion sequence, (b) Segmentation of mixed walking/running sequence. Top graph shows 
correct segmentation (dotted red line) and estimated segmentation (solid blue line). 
Bottom graph depicts the segmentation of the estimated LDS states. 



To test the classification ability of our learned model we next considered 
segmentation of sequences of complex motion, i.e., motion consisting of alter- 

® We explored SLDS models with two to six states. 
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nations of “jog” and “walk.”^ Identification of different motion “regimes” was 
conducted using the approximate Viterbi inference. Estimates of “best” switch- 
ing states (st) indicated which of the two models can be considered to be driving 
the corresponding motion segment. One example of this segmentation is depicted 
in Figure 3(b). 

Classification experiments on a set of 20 test sequences gave an error rate® 
of 2.9% over a total of 8312 classified data points. 

Additional classification experiments were performed using the structured 
variational inference technique. Figure 4 depicts state estimates and variational 
parameters in iterations 1 through 4 of variational inference. Initial uncertain 
switching state distribution (s) leads to low variational state noise variance 
Q (whose determinant is indicated by \Qv\ in Figure 4) and low variational 
state transition matrix ( whose determinant is indicated by \A^\ in Figure 4). 
Through further iterations the variational inference algorithm converges to the 
true switching state sequence. 

4.2 Tracking 

A second experiment explored the utility of the SLDS model in improving track- 
ing of the human figure from video. The difficulty in this case is that feature 
(joint angle) measurements are not readily available from a sequence of image in- 
tensities. Hence, we use the SLDS as a multi-hypothesis predictor that initializes 
multiple local template searches in the image space. Instead of choosing multi- 
ple hypotheses at each time step we pick the best S hypothesis with the 

highest switching probability, i.e., where il = argmaxj 

Given the predicted means for the figure locations, state-space observations 
are obtained by local image registration, or hill-climbing. This identifies the 
state-space modes in the likelihood function given by the template model. A 
larger set of measurements could be explored through sampling, as described 
in [6]. Given these observations of figure state, the regular SLDS filtering yields 
SLDS state priors. 

Figure 5 shows stills from a representative example of SLDS tracking of walk- 
ing motion. In this experiment, simple template features were used to model the 
appearance of the figure. Each link in the model has an associated template, 
which is initialized manually in the first frame and applied throughout the se- 
quence. Template features are not robust to appearance changes such as lighting 
effects or the wrinkling of cloth. As a result, a template-based tracker can benefit 
substantially from an accurate dynamical model. 

A constant velocity predictor does poorly in this case, leading to tracking 
failure by frame seven (shown in Figure 4.b). The learned SLDS model gives 
improved predictions leading to more robust tracking. 

^ Test sequences were constructed by concatenating in random order randomly se- 
lected and noise corrupted training sequences. Transitions between sequences were 
smoothed using B-spline smoothing. 

® Classification error was defined as the difference between inferred segmentation and 
true segmentation accumulated over all sequences, e = I “ '^true t\ ■ 
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(c) Iteration 3 



(d) Iteration 4 



Fig. 4. Iterations of variational inference. The graphs depict: continuous state esti- 
mates (x), switching state estimates (s), HMM variational parameter logg, determi- 
nants of LDS variational parameters \Qv \ and \A^\, and the true (dotted) and estimated 
measurements E[y], 



4.3 Synthesis and Interpolation 

In Section 2 we introduced SLDS as a generative model. Nonetheless, SLDS is 
most commonly employed as a classifier (e.g. Section 4.1.) To test the power 
of the learned SLDS framework we examined its use in synthesizing realistic- 
looking motion sequences and interpolating motion between missing frames. 

In the first set of experiments the learned walk/jog SLDS was used to generate 
a “synthetic walk.” Two stick figure motion sequences of the noise driven model 
are shown in Figure 6. Depending on the amount of noise used to drive the 
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Fig. 5. (a) Tracker (in white) using constant velocity predictor drifts off track by 
frame 7. (b) SLDS-based tracker is on track at frame 7. Model (switching state) 3 has 
the highest likelihood. Black lines show prior mean and observation, (c) SLDS tracker 
at frame 20. 




Fig. 6. Synthesized walk motion over 50 frames using SLDS as a generative model. 
States of the synthesized motion are shown on the bottom. 



model the stick figure exhibits more or less “natural” -looking walk. Departure 
from the realistic walk becomes more evident as the simulation time progresses. 
This behavior is not unexpected as the SLDS in fact learns locally consistent 
motion patterns. 

Another realistic situation may call for filling-in a small number of missing 
frames from a large motion sequence. SLDS can then be utilized as an interpola- 
tion function. In a set of experiments we employed the learned walk/jog model to 
interpolate a walk motion over two sequences with missing frames (see Figure 7.) 
The visual quality of the interpolation and the motion synthesized from it was 
high (left column in Figure 7.) As expected, the sparseness of the measurement 
set had definite bearing on this quality. 

5 Conclusions 

We have introduced a new approach to dynamics learning based on switching 
linear models. We have proposed two approximation techniques, Viter bi and 
structural variational inference, which overcomes the exponential complexity of 
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(a) 



(b) 




(c) 



(d) 



Fig. 7. SLDS as an interpolation function. A motion sequences with missing measure- 
ments between frames 50 and 100 was interpolated using an SLDS model. Symbols ’x’ 
in figure (a) indicate known measurement points. Solid lines show interpolated joint 
angle values. Dotted lines indicate ground truth (smoothing with no missing measure- 
ments.) Figure (c) depicts corresponding SLDS states. Stick figure motion generated 
from interpolated data is shown in figure (d). Figure (b) shows true stick figure motion. 



exact inference. Simplicity of approximate Viter bi inference is contrasted by the 
lack of an exact bound on the approximation error. This is a problem in general 
with greedy Viterbi-style approximations, as well as with Markov chain Monte 
Carlo methods [20]. On the other hand, more complex structured variational 
inference guarantees minimization of this error by considering global approxi- 
mation of intractable SLDS distribution. 

Our preliminary experiments have demonstrated promising results in clas- 
sification of human motion, improved visual tracking performance, and motion 
synthesis and interpolation using our SLDS framework. We demonstrated accu- 
rate discrimination between walking and jogging motions. We showed that SLDS 
models provide more robust tracking performance than simple constant velocity 
predictors. The fact that these models can be learned from data may be an im- 
portant advantage in figure tracking, where accurate physics-based dynamical 
models may be prohibitively complex. 
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We are currently building a more comprehensive collection of frontoparallel 
human motion. We plan to build SLDS models for wide variety of motions and 
performers and evaluate their performance. 
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Abstract. This paper includes an application consisting of an auto- 
matic gear-box and cruise controller which naturally is modelled as a 
hybrid system including state jumps in the continuous state of the con- 
troller. Motivated by this application, we extend existing stability results 
to include state jumps as well. The proposed stability results are based on 
Lyapunov techniques. The search for the (piecewise quadratic) Lyapunov 
functions is formulated as a linear matrix inequality (LMI) problem. It 
is shown how the proposed stability analysis is applied to the automatic 
gear-box and cruise controller. 



1 Introduction 

Many physical systems today are modeled by interacting continuous and discrete 
event systems. Such hybrid systems contain both continuous and discrete states 
that influence the dynamic behavior. There is a lot of interest in these kind of 
systems today, since a large number of systems are neither pure continuous nor 
discrete but a combination. This is mostly due to the growing use of computers 
in the control of physical plants but also as a result of the hybrid nature of many 
physical processes. Physical systems suitably modeled as hybrid systems are for 
instance the management of a fishery resource [13], computer disk system [9], 
motion control systems [5], robotics [6], power systems [10], systems in classical 
mechanics [4], air traffic management [18] and automated vehicles [12]. 

This paper includes an application consisting of an automatic gear-box and 
cruise controller. Both the automatic gear-box (plant) and the cruise controller 
(controller) are naturally modeled as hybrid systems which interact to control 
the velocity at a desired value. The automatic gear-box is modeled as simple as 
possible, with the velocity and the gear position as continuous and discrete state 
respectively. The discrete state is changed when the velocity reaches different 
values, which affects the continuous dynamics. The cruise controller consists of 
a Pl-controller, where the continuous state is the integrator state, implying that 
the velocity converges to the desired state despite the influence of disturbances. 
To obtain a comfortable ride, there are restrictions imposed on the derivative of 
the acceleration. This implies that the gain in the controller must have differ- 
ent values for different gear positions, implying that it acts as a discrete state 
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with different values. Furthermore, the restrictions also imply state jumps in 
the integrator state at the times when the gear position is changed (bumpless 
transfer) . 

Many stability results for hybrid systems using a Lyapunov approach have 
been proposed in the literature; see for instance [14, 7, 3, 21]. However, none of 
the approaches considers the additional complexity to include also state jumps 
in the analysis. Therefore, stability results applicable to hybrid systems with 
state jumps are proposed in this paper. The problem to find the different lo- 
cal (piecewise quadratic) Lyapunov functions is formulated as a linear matrix 
inequality (LMI) problem [2] , for which there exists numerical software [8] . 

The proposed stability results in this paper can be generalized to consider 
even larger classes of hybrid systems than the application model. Such results 
have in fact been carried out in the Ph.D. thesis [15]. However, to keep the paper 
short and reduce the complexity, possible generalizations will not be discussed 
herein but we refer to the thesis for interested readers. 

The outline of this paper is as follows: The application is given in detail in the 
next section motivating the use of stability results for hybrid systems including 
state jumps. Section 3 proposes conditions for exponential stability. It is shown 
how the stability result can be formulated as a linear matrix inequality (LMI) 
problem in Section 4. The paper is concluded by showing how the proposed 
stability result is applied to the gear-box application. 

2 Application and Hybrid Model 

The proposed theory in this paper is motivated by the following application: 



2.1 Gear-Box Application 



A motor together with transmission through a gear-box is naturally modeled by 
continuous and discrete states. Nonlinear models describing the dynamics of a 
vehicle with throttle angle as input are given in [19]. Continuous state variables 
are the manifold pressure and velocity of the vehicle, and a discrete state variable 
is the gear position. In this example, a satisfactory model illustrating the hybrid 
behavior is obtained by assuming that the input signal is the torque T out 
from the motor (hence, all dynamics in the motor are neglected). The gear-box 
transforms the torque T and angular velocity u) according to 



Ti = pT and u>i = -uj, 
P 



( 1 ) 



where T\ is the torque and u)\ is the angular velocity of the wheels, and p is the 
gear position. If the radius of the wheel is r, the force F accelerating the vehicle 
and the velocity v of the vehicle becomes 



F = Ti/r and v = rwi. 



( 2 ) 
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The vehicle acceleration is according to Newton’s law of motion 

Mv = F-Fi, (3) 

where M is the weight of the car and Fi is the load force induced from the road. 
If it is assumed that Fi is proportional to the square of the vehicle velocity v 
and the road angle is a, this force can be modeled as 

Fi = fcw^sign V + Mg sin a. (4) 

By combining (1), (2) and (4) into (3), the vehicle dynamics is given by 

^ ^ “ 10-t'^sign u - g sin a, 

UJ = PrV, 

where Pr = p/r is assumed to take values in the discrete set {pri,Pr-iiPri,Pri\, 
Pri > Pr 2 > Pr 3 > Pr^- Hence, there are four possible discrete gear positions, 
where Pn corresponds to gear 1, p^2 to gear 2 and so on. 

In this illustrative example, the automatic gear-box is designed in such a 
way that the change of gear occurs if the engine rotational speed exceeds oJhigh, 
implying a higher gear (if not already gear 4), or goes below ujiow, implying 
a lower gear (if not already gear 1). Depending on the gear, the values uiMgh 
and uiow corresponds to different velocities of the vehicle; see (5). The desired 
behavior is obtained by changing gear position at velocities given by the switch 
sets 

G I U — ^high\ and G 3^^ | U — ^low} D — ^5 

Pri Pvi+i 

where S'iy+i denotes gear position changes from z to f -I- 1 and vice versa for 

The cruise controller is designed in the following way. The torque T consists 
of the terms 

k 

T = Tp + T[-\ u^signu (6) 

Pr 

where 



Tp — Kj~{Vref 

Tl = ^{Vref -V), (7) 

and Vref is the desired velocity. Hence, the cruise controller (6) is essentially 
a Pl-controller which compensates for the nonlinearity due to the load force. 
If the closed-loop system is (asymptotically) stable, the integrator part of the 
controller implies that the vehicle velocity v converges to the desired velocity 
for stationary input values Vref despite the influence of a constant road angle a. 
Every time a new value of the desired velocity Vref is given by the driver, the 
integrator state T/ is put to zero. 
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Besides stabilizing the closed-loop system, the parameters and should 
be selected in such a way that a desirable performance is obtained. A comfortable 
ride is maintained if the acceleration is limited to lul < 2m/s^; cf. [1]. This 
condition restricts the gain K^. In the design of the integration time there is 
a tradeoff between fast convergence and small overshoot. 

Besides conditions on the acceleration, it is desirable also to have restrictions 
on the derivative of the acceleration since abrupt changes of this variable can be 
quite uncomfortable. One reason for possible abrupt changes of v occurs when 
the gear position is changed. If tk denotes the time when the change of gear 
occurs and and denote the times just before and after that time, and Kr 
takes values in the set {Kr^, Kr^, Kr^, Kr^}, where corresponds to gear 1, 
and so on, then (5) and (7) imply that there are no abrupt changes of v due to 
a change of gear if 

Priori — Pri^\^ri^\ 

) = Pn+iTXtfc ) gear z to z -|- 1 z = l,2,3. (8) 

Pn+iTi{t~) = pr,Ti{t^) gear z -|- 1 to z 

Hence, by designing the gain parameters j • ■ • , Kr^ and abruptly changing the 
value of the T/-variable such that (8) is satisfied, discontinuities in zi (and hence 
v) due to change of gear position are avoided. The jump in the state variable 
T/ avoiding jumps in the control signal T is commonly called bumpless transfer 
[11]. 

Let the numerical values be equal to: Pn = 50, p^2 = 32, = 20, Pn = 14, 

k = 0.7, M = 1500, g = 10, uiiow = 230, Uhigh = 500, = 3.75, = 5.86, 

= 9.37, = 13.39, Tr = 40, Vref = 30, m(0) = m3, u(0) = 14 and T/(0) = 

0. For a specified desired velocity Vref the system converges exponentially to Vref, 
which will be verified by LMIs after the stability theory. 

2.2 Hybrid Model 

The hybrid model in the application has the form: 

X = A(m)x, 

a;+ =Tp(x,m), (9) 

77Z+ = (j>(x, m), 

where x € 3?” is the continuous state and m G Ai = {mi,...,mAr} is the 
discrete state. The hybrid state space H is the Cartesian product 3?” x Al. 
The continuous dynamics is given by a linear differential equation, including the 
possibility of expressing state jumps by the function tp : W x Ai ^ 3?", and 
(j) : 3?” X A4 ^ A4 is a function describing the dynamics of the discrete state. 
The notation x~^ and mA means the next state of x and m respectively. The 
hybrid system described in (9) is autonomous, i.e. there are no external inputs 
affecting the dynamics. This may be the result when external inputs are feed 
back functions of the continuous and discrete state, which is the case in the 
application. 
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The discrete state changes when x and m take certain values can, instead of 
being described by a function be expressed by a number of switch sets Sij 
(as in the application), which are related to (j) according to 

= {a; G 3?” I ruj = (j>{x,mi)}, i G In, j G In, 

where N is the number of elements in Ai, and In = {1, 2, . . . , A^}; cf. [20, 
17]. Hence, the switch sets indicate where in the continuous state space 3?” the 
discrete state nn changes to nij. It is usual to specify only those switch sets 
that cause a change of discrete state variable from rrii to mj where rrii yf nij. 
The switch sets are often given as geometrical hypersurfaces or hyperplanes (as 
in the application). Similarly, the set of states where state jumps occur for the 
different discrete states can equivalently be described by sets 

Ji = {x G 3?” I x"*" = 'il^{x,m,i)}, i G In- 

It is assumed that Sij and Ji coincide in this paper and the next continuous 
state is related to the previous one according to 

x"*" = G{mi)x (10) 



at these states. 

In the stability analysis given next, it is assumed that there only is a finite 
number of switches in finite time. Hence, the continuous and discrete dynamics 
is well behaved. 



3 Exponential Stability 

We are now prepared to show exponential stability of hybrid systems including 
state jumps. 



3.1 Region Partitioning 

To show stability, 17 C of the hybrid state-space is partitioned into I disjoint 
regions. If for a given initial point in 17, fc = 1, 2 . . . are the consecutive times 
when the trajectory passes from one region to another, it is assumed that the 
partitioning is made in such a way that tk is strictly less than tk+i, i.e. tk < tk+i- 
Let 17 be a hybrid set. Then, the following projection sets are defined: 

17“ = {x G 3?” I (x,m) G 17}, 

17“’”"“ = {x G 3?” I (x,m*) G 17}, (11) 

17™ = {to G Ad I (x,to) G 17}. 



Hence, 17“ and 17“’™* are sets consisting of continuous states while 17™ consists 
of discrete states. 
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Assume that a trajectory satisfies (9) for any initial value in Hq. Let e > 0 
andefine the neighboring regions q & h, r G Ii, {q ^ r) by 

Aq^r = {{x,m) G n \ 3t > 0 such that {x{t — e), m{t — e)) G Hq and 
{x{t + e), m{t + e)) G f^r, when £ ^ 0}, 

which are sets where trajectories pass from f2q to f2r- Let 

I A = {{q,r) I Aq^r + 0}, 

which is the set of tuples indicating that there is at least one point for which the 
trajectory passes from Qq to Qr- 

Let Vq = x"^PqX, q G le, be a quadratic function which is used as a measure 
of the system’s (abstract) energy in region f2g. Let the overall energy be defined 
as 

V{x) = Vq{x) when (x,m) G Hq, (12) 

which, in general, is a discontinuous function at the neighboring regions Aq^r, 
{q, r) G I A- Since it is assumed that the partitioning is made in such a way that 
tk < tfc+i for every trajectory with initial point in f?, it is ensured that the 
overall energy defined in (12) is piecewise continuous as a function of time. The 
time derivative of Vq (x) in region Hq can be written as: 

Vq{x) = A{mifPq + PqA{mi), X G f2^q^\ m, G 12™, 

using the projection sets in (11). 

3.2 Exponential Stability Conditions 

Definition 1. The region of exponential attraction R{ki,k 2 ) of a hybrid sys- 
tem (9) is the set of initial hybrid states for which the continuous trajectory 
exponentially converges to the origin according to 

R{kx,k 2 ) = {(xo.TOo) G Ho \ ||a;(t)|| < kie~^'^*'\\xo\\, t > 0, > 0,^ > 0}. 

Exponential stability in a region can be verified by the following theorem. The 
proof of this theorem can be found in [16, 15]. 

Theorem 1. If there exist Pq, q G Ii, and constants a > 0 and (3 > Q, such 
that 

1. x G Hq, ax'^x < x'^PqX < fjx'^x, q & Ii 

2. x £ x'^{A{miY'Pq + PqA{mi))x < —x'^x, rrii G 12 ™, q £ h 

3. x £ Aq^^, x'^G{mi)^ PrG{mi)x < x"'- PqX, mi£l2^, (q,r) £ Ia 

then the equilibrium point 0 is exponentially stable in the sense of Lyapunov. If 
the assumptions hold globally, then the equilibrium point 0 is globally exponen- 
tially stable. 
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IfV is defined as in (12), then 

Rc = {{x,m) G H I V{x) < c} C R{ki,k 2 ), 



where Rc C Q and 




(13) 



The left-hand side of the first condition guarantees that each local quadratic 
Lyapunov function is positive. The righ-hand side is introduced to calculate 
the upper bound of the exponential convergence rate. The second and third 
condition guarantee that the overall energy V (12) decreases, both in regions 
(second condition) and when another region is entered (third condition). 

To show exponential stability, the existence of Pq’s satisfying the stability 
conditions has to be verified. This can be done by solving an LMI problem, 
described next. 



4 Linear Matrix Inequalities 

All conditions of the stability theorem are constrained to be satisfied, not in the 
entire state space but in a part of the continuous state space. The first condition 
is restricted to the region 17®, the second condition is restricted to I?®’™* and 
the third condition is restricted to It is now described how the constrained 
conditions can be replaced by unconstrained conditions, by first expressing the 
regions by positive (quadratic) functions and then using a general technique 
called the 5-procedure to obtain an unconstrained condition. This procedure is 
first explained in general terms and then applied to the constrained conditions 
in the stability theorem. 

4.1 Prom Constrained Conditions to Unconstrained Conditions 

Replacement of Constraint to Regions with Constraint by Functions. 

Assume that F^{x) : 3?” — > 3? is a function having unknown variables which are 
to be decided, satisfying the condition 

F^(x) > 0 for all x in the region TZ. (14) 

Assume that F'^{x) : 3?” — > 3?, k G R, are known functions satisfying 

F^{x) >D,kG R, for all x in the region TZ. 

Condition (14) can then be replaced with the possibly stronger condition 

> 0 for all x satisfying F^(x) >Q,kG R. (15) 

Hence, the condition F'^{x) > 0 constrained to the region TZ has been replaced 
by constraints by the functions F^(x) > Q, k G R. The replacement of TZ by 
functions F^{x) > 0 is illustrated in Figure 1. 
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Fig. 1. The shaded region TZ is replaced with a region described by all x satis- 
fying F^{x) > 0 and F^{x) > 0, which is the dashed region. 

<S-procedure. It is possible to replace the constrained condition (15) by a 
condition without constraints by introducing additional variables > 0, fc € /« 
in the following way: 

Lemma 1. [2] If there exist > 0, k G /«, such that 

K 

Mx G SR", F°{x) > X’^F’^ix), (16) 

fc=i 

then (15) holds. 

The proof follows directly by noting that the right-hand side of (16) is greater 
or equal to zero for all x satisfying F^{x) >0,kG 1^, since > 0, fc G /«• 

The constrained condition (14) has been replaced by the unconstrained con- 
dition in Lemma 1. In the case of quadratic functions 

F^{x)=x^Q^x, k = 0,...,K, (17) 

where = (Q^)^ G 3?" x 3?", the condition (16) can be written as an LMI: 

K 

Q°>^A'=Q^ A'^^O, fcG/„, (18) 

fc=i 

The finesse of formulating the LMI condition as (18) instead of only > 0 is 
that the condition x"^Q^x > 0 does not have to be fulfilled in the entire state 
space, implying that has to be positive semi-definite, but only in a part of 
the state space where at least all x'^Q^x > 0 are satisfied. The unknown matrix 
variable and the different A^ can be found by solving the LMI problem (18). 
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Some remarks should be made. First, it is always possible to replace an 
arbitrary region TZ with larger region constraints expressed by quadratic forms 
F^{x) > 0, by simply letting F^{x) be positive semi-definite functions. This 
implies that F^{x) > 0 has to be satisfied in the entire continuous state space. 
However, one should avoid replacing a region with quadratic forms F^{x) > 0 
such that the states satisfying all these inequalities is much larger than TZ, since 
this conservatism may imply that the stronger condition (15) does not have a 
solution although (14) has. In some cases, the conservatism is no problem since 
a solution will exist anyway (as in the application). However, in other cases, 
not being too conservative is crucial for a solution to exist. In Section 4.3 it is 
explained more thoroughly how to specify the parameters in the quadratic forms 
(17) such that regions TZ can be replaced by quadratic forms F^{x) > 0 without 
too much conservatism. 

Second, the replacement of (15) by Lemma 1 may also be conservative. How- 
ever, it can be shown that the converse is true in the case of a single quadratic 
form, K = 1 [2], provided that there is some x such that F^{x) > 0. 

Third, in the case of hypersurfaces defined by F^(x) = 0, fc G it is not 
necessary to require that the different A^, fc G /«, have to be greater or equal to 
zero in Lemma 1, since this lemma holds despite the sign of these constants. 

The above procedure is now applied to the constrained conditions in the 
stability theorem. 



Stability Conditions. All conditions of the stability theorem are described 
by F'^{x) > 0, where F'^{x) is a quadratic function defined as in (17). The first 
and second conditions in the stability theorems are restricted to regions 17® and 
respectively. These conditions can be replaced by unconstrained conditions 
of the form (18). Matrices corresponding to regions 17® are denoted and 
regions 17®’™* are denoted Qq^rm- 

The third condition is restricted to hypersurfaces A®,,. When these are given 
by F^{x) = 0, fc G Ik, where each F^{x) has the form (17), there will be 
no restrictions on the additional variables in (18). However, if some switch 
surface cannot exactly be described by F^{x) = 0, fc G /«, then it is possible to 
include such a region with quadratic functions satisfying F^{x) > 0, in which 
case the additional variables A^ in (18) have to be greater or equal to zero. 
Matrices corresponding to hypersurfaces A® ,, are denoted Qg ,.- 

4.2 LMIs for Hybrid Systems with Linear Vector Fields 

In the case of verifying exponential stability, it may be desirable not only to 
find a solution but to search for a solution that gives a better estimate of the 
convergence rate k 2 in (13). This can be achieved by searching for a solution 
where [3 is minimized. The LMI problem then becomes as follows: 
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LMI Problem If there is a solution to 
min/3 subject to 

0. a > 0, > 0, > 0, ^. > 0, q€ h 

2. A{m,fPg + P^A{mi) + Y117' h 

3. G{miY PrG{mi) + Y!k=l 'nq,rQ\,r < A. («> ^ I A 

then the equilibrium point 0 is exponentially stable in the sense of Lyapunov. 

The variables a, iiq, Vq, ’dq^tm and matrices Pq are unknowns, while the differ- 
ent Q:s are known matrices corresponding to the different local regions where 
the conditions have to be valid. The convergencerate rate is estimated as in 
Theorem 1. 

The LMI formulation can be extended to consider affine vector fields by ex- 
tending the quadratic local Lyapunov functions from Vq{x) = PqX to Vq{x) = 
Tiq + 2p^x + x^PqX. Nonlinear vector fields may also be handled by slightly 
modifying the second condition in the LMI problem [15]. 

4.3 Describing Regions by Quadratic Forms 

It is now explained how to specify the parameters in the quadratic forms (17) 
such that the set of states satisfying all quadratic forms F^{x) > 0 includes 
the set TZ. We are focusing on regions partitioned by hyperplanes. More general 
regions are discussed in [15]. 



Quadratic Forms Describing Half-Planes. If a region TZ containing the 
origin is given by the set of states restricted by two half-planes 

(c“)^x > 0 and {c^)^x > 0, 

then TZ will be described by a quadratic form 

x'^Q^x > 0, where = c“(c^)^ -I- c*'(c“)^. (19) 

The set of states satisfying a quadratic form (19) has the property that if x\ 
satisfies the inequality so does — xi; see Figure 2. 

If the dimension n is equal to two, there is no reason for replacing a region TZ 
by a quadratic form (19) described by more than two hyperplanes, since the set 
of states satisfying several half-planes can equivalently be described by only two 
half-planes. However, this is reasonable in higher dimensions. In this case, the 
quadratic forms x"^Q^x > 0 are obtained by taking all possible combinations of 
two different half-planes. This results in different quadratic forms as in 

(19) and hence also variables in (16), where g is the number of half-planes. 
There is no reason to add the combinations of the same half-planes since these 
quadratic forms are greater or equal to zero for all states. 




Stability of Hybrid Systems Using LMIs - A Gear-Box Application 391 





Fig. 2. (a) Region (dashed) TZ restricted by hyperplanes (c“)^a; > 0 and 
(c*')^x > 0. (b) Region (dashed) of states satisfying x^Q^x > 0, where 
Qi = c“(c*')^ -bc''(c“)^. 



The reason for specifying a number of quadratic inequalities instead of only 
one, in case TZ is restricted by several half-planes, is that TZ cannot exactly be 
described by the set of states given by a quadratic form greater or equal to zero, 
even if TZ is symmetric around the origin (meaning that ii x ^TZ then —x € TZ] 
cf. Figure 2b). The set of states satisfying x'^Q^x > 0 for a quadratic form given 
by any combination of two half-planes describing TZ will be strictly larger than 
TZ. Since it cannot be said that the set of states given by one quadratic form 
x"^Q^x > 0 is better than another, all reasonable combinations are specified. 
The variables obtained by solving the resulting LMI problem (18) then decide 
the quadratic form such that TZ most suitably is replaced by the set of states 
satisfying the quadratic inequality. 



Quadratic Forms Describing Hyperplanes. The quadratic forms equal to 
zero at a hyperplane can be obtained as follows. Assume that TZ is given by the 
set of states satisfying a hyperplane 

c^x = 0, (20) 

where c = [c^ . . . G 3?”. The states satisfying (20) also satisfy 

2{\^x)^{c^x) = 0 ( 21 ) 

where A = [A^, . . . , A"]^ G 3?" are arbitrary additional variables. The equality in 
(21) can be written as 

n 

+ x^cA^x = ^ X^x^Q^x = 0 




5 Stability of the Gear-Box Application 

We are now prepared to show stability of the gear-box application. By denoting 
Av = Vref — V and ATj = T[, the closed-loop dynamics becomes 

Av —prKr/M —Pr/M Av 

Afi\ ^ [ Kr/Tr 0 J [aTi\ ’ 

where M = 1500, = 40, pr G {50,32,20,14}, Kr G {3.75,5.86,9.37,13.39} 

and PrKr = 187.5 for all discrete states. For a specified desired velocity Vref 
(= 30 m/s) the system converges exponentially to Vref, illustrated in Figure 3, 
and formally proven next. 

If it is first assumed that there are no state jumps in T/, stability can be 
shown by a single partitioning, implying a single Lyapunov function common for 
all discrete states. This results in a solution 

{255.589 72.262' 

^ ~ [ 72.262 40.822 

satisfying the conditions in the LMI problem. Hence, the hybrid system is glob- 
ally exponentially stable without state jumps. The optimal value of /? = 277.6388. 

If the state jumps are included in the dynamics, they occur when the discrete 
state is changed. Trajectories satisfying the condition Tj > Kr{v — Vref) cross 
the switch set S'iy+i and from left to right (see Figur 3) and oppositely 

for Ti < Kr{v — Vref)- In the operating region of this cruise controller (7/(0) is 
always put to zero when a new desired velocity is given), the gear shiftings will 
always occur from lower to higher gear when the first condition is satisfied and 
conversely in the second case. Hence, the third condition of the LMI problem is 
formulated such that the energy decreases passing from gears z to / -I- 1 satisfying 
T/ > Kr{v — Vref) and gears z -I- 1 to z satisfying Tj < Kr{v — Vref)- 

Consider the case when the trajectories start in the first region. The jump 
condition (8) on the form (10) gives 

[l 0 1 

G{mi) = Q Prj+i z= 1,2,3. 

p<~i 

According to the LMI problem, the third LMI condition then becomes 

G{niif' PG{nii) < P, 
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which is equivalent to 

p"’2(^)2<p2.2^ 

Pvi 



where is the (2, 2) element of P. Since < 1, the energy will decrease 

due to the state jumps for any quadratic function Px. Therefore, the same 
solution as above verifies stability in this case. However, when the trajectories 
start in the second region, the jump condition (8) is the same as above except 
that Pri and Pri+i change position. In this case, there will not exist any solution 



since 



( Pr, 



'Pr. 



-r>i. 



To overcome this problem, the state space is further partitioned to verify 
exponential stability. One quadratic candidate Lyapunov-like function is associ- 
ated with each of the discrete states. The switch surfaces then coincide 

with for i = 1,2, 3. Solving the LMI problem leads to a solution 





304.082 87.089 






248.013 79.625 


Pi = 


87.089 376.934 




, P 2 = 


79.625 144.215 




'212.101 59.328' 






'147.495 53.571' 


P 3 = 


59.328 53.788 




5 ^4 = 


53.571 24.112 



Hence, the hybrid system is exponentially stable also in the case of state jumps. 
The optimal value of /3 = 439.9. The level curves for the local quadratic Lya- 
punov functions are shown in Figure 3. 



6 Conclusions 

A gear-box application has served as a motivation for investigating stability of 
hybrid systems including state jumps. None of the results reported in the litera- 
ture can deal with this additional complexity. Stability results for hybrid systems 
including state jumps are proposed in this paper using Lyapunov techniques. It 
has been shown how to formulate the search for the (piecewise quadratic) Lya- 
punov functions as a linear matrix inequality (LMI) problem. The theory has 
been applied to the gear-box application to formally show stability. 
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Abstract. A special class of hybrid systems, that occurs in many ap- 
plications, are the piecewise linear systems. Due to their nonlinearity, 
they may often be difficult to analyse. Therefore, different approximat- 
ing methods have been developed for analysis, verification and control 
design. This paper considers one such method, and gives a method for 
investigating how sensitive it is to changes in the dynamics of the under- 
lying linear subsystems. This method can be used either for robustness 
analysis or for control design. 



1 Introduction 

Piecewise linear systems constitute a special class of hybrid systems. These sys- 
tems consist of several linear (or rather affine) subsystems, between which switch- 
ings occur at different occasions. In this paper, the piecewise linear systems will 
be on the form 



X = A^x + , X & , i = 1, . . . , N . (1) 

This implies that the dynamics of a trajectory x{t) just depends on x, not on t 
or on any external input. The different regions A* are assumed to be polyhedra, 
i.e., regions defined by linear inequalities. Systems of this kind occur in many 
applications. A very simple example of a piecewise linear system could be a linear 
system, controlled by linear feedback, but where the control signal is bounded. 

Since piecewise linear systems, like other hybrid systems, are highly non- 
linear, they might be difficult to analyse. Several approximating methods for 
analysis, verification, and control design have therefore been developed for dif- 
ferent classes of hybrid systems, e.g., [2, 3, 4], [5], [7], [8], [10]. 

The problems considered in this paper arise for example when considering 
robustness aspects of the method proposed in [5], which is a method for veri- 
fication of piecewise linear switched systems. In this method, the behaviour of 
the vector field x{t) at the borders of the regions A* is analysed. Specifically, 
questions such as “At a given face of the polyhedron A*, is there a point, xq, 
such that xo is pointing out of A*, or are all trajectories at this face going into 
A*?” are answered (this kind of computations has also been used by others, 
e.g., by [6]). The information obtained is used to determine which transitions 
between different regions are possible, which transitions are guaranteed to occur 
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non-deterministically (i.e., one transition out of a set of transitions from a given 
polyhedron is guaranteed to occur) and which are not. Then finite automata 
are constructed, showing the guaranteed or possible transitions. The finite au- 
tomata give an approximation of the system, and can be used for different kinds 
of verification. For example, we can guarantee that certain states in the original 
system are not reachable from some other initial states, by proving that there 
is no sequence of possible transitions in the finite automata, taking the system 
state from the region of the initial states to the region of the final states. 

Like all other methods mentioned above, the method in [5] assumes that a 
model of the system is given. It would be desirable to be able to determine how 
sensitive the approximating automata are to changes in the underlying linear 
subsystems. Such information could be used to get a measure of how robust the 
verification process is to model errors, or as an aid in a control design process, if 
we would like to adjust the system dynamics without losing the verified property. 
Sometimes we would only be interested in that some crucial transitions should 
not change, whereas in other cases we might want the entire approximating 
automata to remain invariant. 

Since the approximating method considers the behaviour of x(t) at the bor- 
ders of the regions Ai*, we must determine how this behaviour changes with 
varying A® and 6*. That is the topic of this paper. 



2 Notation and Problem Formnlation 

The systems considered in this paper are on the form 

x = {A^ - A^)x + b^ -5\ xeX\ , ( 2 ) 

where A* G 6* G M” and AT* c M” are given, while Z\* G M”^", 6^ G M" 

can be viewed either as uncertainties in the model, or as matrices of our choice. 
To begin with, the regions X* will be polytopes, i.e., they are bounded regions 
defined by 



a:* = {x G M" I C^x ^ d*} , (3) 

where C* G M'" m* > n, and ^ denotes componentwise inequality. The case 
of unbounded regions is considered in Sect. 3.2. 

We will use the notation for the Zth row of C*, and for example d\ for the 
Zth element of a vector db We will also introduce the notation 

Xi = {x G M” I C^x ^ d\ C\x = dil , (4) 

i.e., XI is the face of the polytope corresponding to equality in the ?th constraint. 

Now consider one of the polytopes, say AT®, and how the trajectories x{t) 
behave inside it. At a given face of the poly tope, say X^, and for given 
and there are three different options for the qualitative behaviour of the 
trajectories (see Fig. 1): 




398 



J. Roll 



1. They are all exiting the polytope. 

2. They are all entering the polytope. 

3. There exists (at least) one point in X^, where the trajectories are parallel to 
the polytope face. In this latter case we may have some trajectories exiting 
the polytope and others entering it through the same face. 

Since the system is linear inside the polytope, the trajectories are smooth, and 
therefore these three cases are the only possible options. The three different cases 
will lead to different approximating automata. An interesting question is: How 
much could and change, without changing the qualitative behaviour at 
each face of the polytope, i.e., without affecting the approximating automata? 
In other words, for what values of Z\° and do the different cases occur? 

In the following section these problems are solved, and an example is given 
in Sect. 6. 




Fig. 1. Three options for the behaviour of the trajectories in the vicinity of a 
polytope face. 



3 Solutions to the Problems 

Since is a normal vector of the poly tope face X^, we can easily see that the 
three cases from the previous section correspond to the three problems 

1. C^i > 0 for all x € X^, 

2. C^i < 0 for all x £ X^, 

3. C^x = 0 for some x £ X^, 

so our task is to find the sets of solutions (in and (5°) to all these problems. 
Let us begin with the first problem. By using (2) we can rewrite it as 

C^[{A° - A°)x + (6° - (5°)] > 0 for all x £ X^ , (5) 

or 

+ 6°) > Cl,{A°x + ,5°) for all x G , (6) 

This last form has a natural interpretation: On the left hand side we have the 
nominal flow through the face of the poly tope, and the right hand side is the 
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part of the flow that is affected by the variable matrices and <5°. What (6) 
tells us is that the variable flow must be made small enough; otherwise we will 
get a total flow in the other direction from what was specified. 

To get a solution to (6) we need to And a direct representation of X^: 

r r 

= {E I e ^ 0, E ■ (^) 

i=i i=i 

Here Vj G IR”, j = 1, . . . r are the corners of X^. In words, the direct represen- 
tation means that we write each point of X^ as a “weighted mean”, or convex 
combination, of the corners. 

Now, the set of solutions is given by 

j = l,...,r}. (8) 

Proof. Obviously, all (Z\°,i5°) in the set above satisfy (6) for the corners Vj of 
X^. To show that the inequality is satisfied for an arbitrary point x G X^, we 
use the direct representation (7): 

r 

C^^iA^x + 6 °) = ^ X,vj + 6 °) = 

i=i 

r 

i=i 

r 

>^A,C'l(Zl%+<5°) = 

i=i 

r 

= C^(Zi°^A,u,+<5°) = 
i=i 

= C°^{A°x + S°) 

□ 

Note that the solution set is a polyhedron in the space IR”^" x IR”, and 
therefore convex. 

The second problem {Cf^x < 0 for all x G Xf^f) is treated in the same way 
as the first. The solution set of the third problem can then be obtained as the 
complement of the first two solution sets. 

3.1 Multiple Requirements 

So far, we have only been looking at one single polytope face. In most cases, 
the requirements may stipulate that several transitions of an approximating au- 
tomaton should remain invariant. This case is easily handled by partitioning the 
problem into subproblems of the form treated above, and then taking the inter- 
section of the solution sets as the solution set for the entire problem. How many 
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transitions we need to consider will depend on the system and what we want to 
verify. For example, if all we are interested in is keeping the state on one side 
of a hyperplane, we only need to consider transitions through this hyperplane. 
It should be noted that considering fewer transitions will lead to a larger - and 
therefore less conservative - solution set, and will also require less computations. 

3.2 Unbounded Polyhedra 

In some cases we might need to consider not only regions that are polytopes, 
but also include the case that is an unbounded polyhedron. This means that 
also could be unbounded. A direct representation of X^ would then be: 

r-\-h r 

= {E I e IR. A,' > 0, E Ai = 1} ■ (9) 

i=i i=i 

As before, vj G K”, j = 1, . . .r are the corners of X^, but in addition we have h 
vectors, Vr+i, ■ • ■ , Vr+h, which are parallel to the unbounded edges of X^. Note 
that Ar+i, . . . , \r+h are not included in the set of \j that should sum up to one; 
they can be arbitrarily large. 

In this case, the solution set S' for the first problem is given by 

= {(A°, <5°) I + 6°) > + <5°), j = 1, . . . , r; 

J = I,..., h} ■ ( 10 ) 

The proof is analogous to the bounded case. 

4 Interpretations 

Perhaps the most obvious interpretation is to view A* and as uncertainties 
due to model errors and/or noise. The algorithm then provides bounds for the 
uncertainties for the requirements of the approximating automata to hold. For 
natural reasons, the bounds may be very asymmetric, indicating that the system 
is more sensitive to certain types of model errors than to others. 

The problem formulation is quite general in that no structure of A* and (5* 
is assumed. If the uncertainty has some structure, we can parametrise A* and (5* 
accordingly, thereby reducing the dimensionality and simplifying the problem. 
For example, if A* = 0 we get a model with additive noise: 

x = A^x + b'-5\ xGX\ . (11) 

It should be noted that this case is much easier to solve than the general one: It 
turns out that it can be reduced to solving two LP problems. For further details, 
see [9]. 

Another parametrisation is used in the example in Sect. 6. In this parametri- 
sation some of the elements of A* and are common to several polyhedra. 
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An alternative interpretation is to consider Z\* and as parameters of our 
choice, to be used for control design. A natural parametrisation would then 
be (5* = 0, where is a fixed vector that depends on the system, 

while we can choose L® freely. In this way we get (piecewise) linear state feedback 
control, and the problem becomes that of finding the linear state feedback vectors 
U that make our system fulfil the requirements on the approximating automata. 

5 Computational Complexity 

From Sect. 3 we know that once we know a direct representation of XJ^, it is 
trivial to divide x M” into the three solution sets for the three different 

problems. Conversely, if we want the solutions to be written as intersections of 
halfspaces (as in (8)), we need to know the direct representation of XJ;^. Therefore 
the computational complexity for this problem is essentially identical to that of 
finding the direct representation. Unfortunately, the number of vectors needed 
in such a representation grows very quickly with the size of the problem. An 
upper bound for the number of corners in a polytope can be calculated in the 
following way: In a corner, n linearly independent faces meet (where n is still 
the dimension of the state space). Since the polytope has m faces, the maximal 
number of corners cannot be larger than ( ™ ) . 

However, if we restrict ourselves to the case where the polyhedra are formed 
by the state space being divided by hyperplanes, it is fairly easy (but still quite 
time-consuming) to calculate the direct representation of all the polyhedra once 
and for all. The total number of corners is then bounded above by ( ^ ) , where 
M is the number of separating hyperplanes. 



6 Example: A Chemical Reactor 

To demonstrate the properties of this kind of problems, we can look at a simple 
example. In [5], a (fictional) chemical reactor is modeled, and a control strategy 
is proposed, after which some properties are verified. Here we assume that some 
of the parameter values are uncertain, and try to determine how large errors can 
be tolerated before the verification is not valid any more. 



6.1 System Model 

A figure of the chemical reactor is shown in Fig. 2. It consists of a tank containing 
a mixture of two fluids. When a certain temperature is reached, an exothermal 
reaction between the two fluids starts, giving the desired product. The temper- 
ature can be controlled by a heater and a cooler. There is also a blender helping 
to mix the fluid. The mixture is provided through an inflow valve. There is also 
a draining valve. The valves can be either open or closed. 

The system model derived in [5] has two continuous state variables: the fluid 
level X\ and the temperature x^- Furthermore, there are six control signals, each 
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Fig. 2. A schematic figure of the chemical reactor 

one taking a value in {0, 1}. They are described in Table 1. It could be worth 
mentioning that is an artificial, uncontrollable signal that indicates whether 
or not the reaction is in progress. 



Table 1. Inputs to the chemical reactor 



Signal 


Interpretation 


Mb 


blender signal 


Mi 


inflow valve signal 


Md 


draining valve signal 


Mb 


heater signal 


Uc 


cooler signal 


Uy 


reaction signal 



The plant dynamics is described by 

X = A{u)x + b{u) , 



where 



A{u) 



-ahUd 0 

0 -(oTi (1 - Mb) + axsMb) 



(12) 



(13) 



b{u) 



bhUi 

bheat'^h “t” ^coolMc “t“ ^reacMr 



(14) 
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Here we will assume that the coefficients in A{u) and b{u) are uncertain, and 
that they are given by 



tth 




1.23- 10-3 






ttTi 




0.15- 10-3 




(5ti 


O-T2 




0.22- 10-3 




to 


6h 


= 


9.838 


- 


Shh 


^heat 




29.43 • 10-3 




*^heat 


^cool 




-44.15- 10-3 




•^cool 


^reac 




44.15-10-3 




^reac 



where the numerical values are the nominal parameter values used in [5]. 

The controller is designed such that the control signals are switched on or off 
when the state reaches certain hyperplanes. The rules are listed below: 

1. Ub = 0 when < 3, Mb = 1 otherwise. 

2. Mi is set to 0 when 25xi + X 2 = 300, and is set to 1 when 25xi + X 2 = 250. 

3. Md = 0 when X 2 < 50, Md = 1 otherwise. 

4. Mb = 1 when X 2 < 50, Uh = 0 otherwise. 

5. Me is set to 0 when X 2 = 110, and is set to 1 when X 2 = 130. 

6. Mr = 0 when X 2 < 50, Mr = 1 otherwise. 



Note that the system contains hysteresis in Mj and Me. This is handled by con- 
sidering each polytope where the hysteresis occurs as two polytopes with two 
different subsystems. 

The switching hyperplanes and an example trajectory are shown in Fig. 3. 
For further details concerning the system model, see [5] . 




Level 



Fig. 3. The switching hyperplanes and an example trajectory. 
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6.2 What to Verify 

There are certain requirements on the controller, which are verified in [5]. These 
are: 

1. The temperature should stay between 0 and 150. 

2. The tank must not be empty, and it must not overflow. The maximum level 
is 13. 

3. There should be an operating region with moderate temperature and fluid 
level which is invariant. In [5], this region is chosen to be 

{a: I 250 < 25a;i + X2 < 300, 110 < xa < 130} . (16) 

4. The operating region should always be reached from the initial states in 

finite time. For simplicity, and to avoid introducing additional conservatism, 
we will not consider this requirement in this paper. 

The requirements can be translated to mathematical formulas: 

1. (a) ±2 > 0 when 0 < < 13, a;2 = 0. 

(b) ±2 < 0 when 0 < xi < 13, X 2 = 150. 

2. (a) ii > 0 when xi = 0, 0 < X 2 < 150. 

(b) ii < 0 when xi = 13, 0 < a;2 < 150. 

3. (a) ±2 > 0 when 250 < 25a;i + X 2 < 300, X 2 = 110. 

(b) ±2 < 0 when 250 < 25a;i + X 2 < 300, X 2 = 130. 

(c) 25 1 a: > 0 when 25a;i + X 2 = 250, 110 < X 2 < 130. 

(d) 25 1 a: < 0 when 25a;i + a;2 = 300, 110 < X 2 < 130. 

We also have to know what linear subsystems x will satisfy in the different cases. 
We get those by considering the control rules. 

6.3 Deriving Bounds for Parameter Uncertainties 

Since we assume that the parameter values are uncertain, the question is how 
large the errors can get before the requirements are violated. By using the algo- 
rithm in Sect. 3, we can get an exact answer to this question: The errors have 
to lie in a polyhedron described by 

0 0 0 1 0 0 0] r 9.8380 ' 

0 0 0 0 1 0 0 0.0294 

0 150 0 0 0 -1 -1 0.0225 

0 0 150 0 0 -1 -1 0.0330 

13 0 0 0 0 0 0 r ,5ah ] 0.0160 

-140 0 -110 25 0 0 1 (5ti 245.7977 

-120 0 -130 25 0 0 1 (5t2 245.8179 

0 0 -110 0 0 0 1 i5bh < 0.0200 . (17) 

-140 0 -110 25 0 1 1 (5heat 245.7536 

-120 0 -130 25 0 1 1 (5cooi 245.7738 

0 0 130 0 0 -1 -1 [,5reacJ 0.0286 

190 0 no 0 0 0 -1 0.2137 

170 0 130 0 0 0 -1 0.1935 

190 0 no 0 0 -1 -1 0.2579 

170 0 130 0 0 -1 -lJ [0.2377 
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We notice immediately that the polyhedron contains the origin, which means 
that the nominal system satisfies the requirements. To get a more intuitive feeling 
for the bounds, one can also consider a subset of the errors and set the other 
errors to zero. For example, suppose that only Oh and axa are uncertain. In order 
not to violate the requirements, their deviations from the nominal values have 
to be contained in the polytope shown in Fig. 4. As we can see, <5 t 2 basically 
has to lie in the interval [—0.18 • 10“^, 0.22 • 10“^], while i5h approximately can 
vary between —1.76 and 1 • 10“^. 




Fig. 4. The allowed region for i5h and assuming that the other errors are 
equal to zero. The right image shows a close-up of the region near the origin. 



7 Conclusions 

We have suggested an approach to investigate how sensitive approximating au- 
tomata for piecewise linear systems, as described in [5], might be to changes 
in the underlying subsystems. Section 3 provided the sets of system matrices 
that satisfy certain demands on the behaviour of the system. As pointed out, 
these can either be seen as giving a measure of how robust the approximating 
automata are to uncertainties in the system, or as giving limits for how much 
the system can be changed, e.g., in a control design process, without altering 
the overall behaviour described by the approximating automata. 

It would be natural to combine these demands with other constraints. For 
example, when using the state feedback parametrisation described in Sect. 4, 
one would probably want to find L* that are optimal in a certain respect. Since 
the solution sets of the two first problems in Sect. 3 are convex, we can form all 
sorts of convex optimisation problems, which can be solved very efficiently once 
we know the direct representations of the polyhedra (see for example [1]). 

The theory in this paper is immediately extendable to switched systems as 
described in [5] . The important thing is that the switch sets are fixed hyperplanes 
in the state space. 
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Abstract. A problem of great interest in the control of hybrid systems 
is the design of least restrictive controllers for reachability specihcations. 
Controller design typically uses game theoretic methods which compute 
the region of the state space for which there exists a control such that for 
all disturbances, an unsafe set is not reached. In general, the computation 
of the controllers requires the steady state solution of a Hamilton- Jacobi- 
Isaacs partial differential equation which is very difficult to compute, if 
it exists. In this paper, we show that for classes of linear systems, the 
controller synthesis problem is decidable: There exists a computational 
algorithm which, after a finite number of steps, will exactly compute the 
least restrictive controller. This result is achieved by a very interesting 
interaction of results from mathematical logic and optimal control. 



1 Introduction 

Reachability specifications for hybrid systems require the trajectories of a hy- 
brid system to avoid an undesirable region of the state space. One of the most 
important problems in the control of hybrid systems is the design of least re- 
strictive controllers which satisfy the reachability specifications. This problem 
has been considered in the context of classical discrete automata [3,15], timed 
automata [1], linear hybrid automata [18], and general hybrid systems [12]. The 
framework presented in [12] has been applied to automated vehicles [11], and air 
traffic management systems [16]. 

Designing least restrictive controllers for reachability specifications requires 
computing the set of all initial states for which there exists a control such that 
for all disturbances, the system will avoid the undesirable region. The least 
restrictive controller is then a static feedback controller which allows any control 
value outside this set of initial conditions while allowing all safe control values 
on the boundary of this set. 

The computation of the safe set of initial states for general hybrid systems 
leads to game theoretic methods, and in particular to the steady state solution 
to Hamilton- Jacobi-Isaacs equations [12]. In general, these partial differential 
equations are very difficult to solve. In addition, steady state solutions, if they 



N. Lynch and B. Krogh (Eds.): HSCC 2000, LNCS 1790, pp. 407—420, 2000. 
© Springer- Verlag Berlin Heidelberg 2000 




408 O. Shakernia, G.J. Pappas and Sh. Sastry 



exist, may be discontinuous even if the initial problem data is continuous. This 
is due to the appearance of shocks, and switchings in the optimal control policy. 

The above difficulties in the computation of least restrictive controllers nat- 
urally raise the following question : Can we find classes of systems where the 
game theoretic approach does not require the solution of the Hamilton- Jacobi- 
Isaacs equation? In this paper, we give a positive answer to the above question 
for normal linear control systems where the system matrix is either nilpotent or 
diagonalizable with purely real rational eigenvalues, and with reachability spec- 
ifications defined by polynomial inequalities. The normality condition requires 
controllability of the linear system with each input and disturbance. This con- 
dition ensures that the optimal control and disturbance are well defined, and 
unique. For the case of real eigenvalues, normality also ensures that the optimal 
control and disturbance have a finite number of switchings [13]. 

Our framework first applies Pontryagin’s maximum principle to synthesize 
the optimal control and worst disturbance. The switching behavior of the control 
and the disturbance is then abstracted by a hybrid system, on which we per- 
form reachability computations. By combining the recent decidability results of 
[8,9], with the normality condition which guarantees finite number of switchings 
[13], we show that the least restrictive controller can be decidahly computed. 
This interesting interplay of results from mathematical logic and optimal con- 
trol presents us with the first decidable controller synthesis problem for classes 
of linear systems. 



2 Controller Synthesis Methodology 

In this section, we briefly review the least restrictive controller synthesis method- 
ology for dynamical systems as presented in [12]. Consider the dynamical system 

x = f{x,u,d) (1) 

with state a; G K”, controls u G U C K”“, disturbances d G D C M"''. Suppose 
there is a target set G C M” which specifies an undesirable region of the state 
space. In the context of dynamic pursuit-evasion games [2,10], the goal of the 
disturbance is to capture the state by driving it into the target set, while the 
goal of the controller is to remain in the safe set G°, the complement of G. The 
target set is described by G = {a; G M” | h{x) < 0}, for a smooth function 
/i : R” ^ M. 

Let U, T> he the set of piecewise continuous functions from M into U and D 
respectively. Given an initial condition xq G M”, input u(-) G U, and disturbance 
d(-) G T>, the flow of the differential equation (1) is a map <l> : MCxUxVxR. — > M” 
given by 



>P{xo,u{-),d{-),t) =xo + 



f{x{T),u{T),d{T))dT. 



( 2 ) 
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Clearly, the largest set of safe initial states for which the controller can avoid 
being captured regardless of the disturbance is given by 

ID = {a;o G K” I 3m(-) G U Vd(-) G D Vt > 0 : u(-), d{-),t) G G"} . (3) 

The set W is called the maximal controlled invariant subset of the safe set 
G°. In the differential games literature, W is called the escape set, since there 
exists a control policy such that the controller can avoid the target set, and is 
called the capture set. While equation (3) conceptually describes the escape set, it 
hardly affords a method of computing it. However, the capturability requirement 
can be encoded by a value function J : M” xU xV x M_ ^ M, which, given an 
initial state a;o G M", u(-) G U, d{-) G T> and t <0, returns 

J{xo,u{-),d{-),t) = h{x{0)). 

Therefore, the value function is the cost of a trajectory that starts at initial state 
xq at time t < 0 and evolves according to system equation (1) with input «.(•), 
disturbance d{-), and ends at final state a;(0) at time t = 0. Since the control tries 
to avoid G while the disturbance tries to steer the system to G, we naturally 
arrive at the dynamic game 

J*(xq, t) = max min J(xq, u(-), d(-),t). 
ueu dex> 

J* is called the optimal value function, since it is the value function correspond- 
ing to the optimal controls and disturbances of the dynamic game. The maximal 
controlled invariant subset of the safe set is described in terms of the optimal 
value function by 



W = {x G M” I min J*(x, f) > 0}. (4) 

t<o 

In order to compute J*{x,t), we first introduce the Hamiltonian 

H{x,p,u,d) = p^ f{x,u,d), (5) 

where p G M” is called the co-state. The optimal Hamiltonian is given by 

H* (x,p) = max min H{x,p,u,d). (6) 

U^U d^D 

The computation of J*{x, f) requires the solution of a modified Hamilton- Jacobi- 
Isaacs partial differential equation [12] 



J*{x, 0) = h{x) 

-^^ = min{0,H*(x,^^)}. 



( 7 ) 



Assuming that (7) has a differentiable solution that converges to a function Jf {x) 
as t ^ — oo, then the set 



IT = {x G M" I Jl{x) > 0} 



(8) 
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is the maximal controlled invariant subset of the safe set G°, and the controller 
g:W^ ^2^ defined by 

^( 3 ,) = I 1“ G C/ I miudeD f{x,u,d)>0^ ifx edW 

[u if a; G U 

is least restrictive controller which renders W invariant [12]. The controller (9) 
is least restrictive in the sense that if gi : K" ^ 2^ is any other controller that 
renders W invariant, then Va; G M” we have gi{x) C g{x). 

The main difficulty in the above framework is the computation of W . In 
general, solving the Hamilton- Jacobi-Isaacs equation (7) seems necessary for 
exactly computing W. However, there are very difficult issues that must be 
resolved in this case: 

1. Existence and uniqueness of solutions, 

2. Existence and uniqueness of steady state solutions, 

3. Shocks: non-smooth solutions to smooth problems, 

4. Convergence of numerical algorithms. 

Given the above difficulties, a natural direction of research is to find classes of 
systems for which some (or all) of these issues are resolved. In this paper, we 
adopt this point of view and we will prove the following theorem. 

Theorem 1 (Decidable Controller Synthesis). Consider the controller syn- 
thesis problem for the dynamical system 

X = Ax Bu Ed (10) 

with controls u G U C M”“, disturbances d G D C and target set G C M” 
given by 

G = {x G M" I /i(a:) < 0}. (11) 

Suppose the dynamical system and target set satisfy the following properties: 

1. A G B G E G 

2. For each column bi of B, the pair (H, 6^) is completely controllable, 

3. For each column Ci of E, the pair {A, ef) is completely controllable, 

4- The feasible sets of controls U and disturbances D are compact rectangles 
with rational vertices, that is U = , U i] and D = 

5. h G Q[xi,X 2 , ■■■,Xn] and fy(a:) yf 0 when h{x) = 0. 

If A is nilpotent or diagonalizable with real rational eigenvalues, then the con- 
troller synthesis problem is decidable. 

Linear systems that are completely controllable by each component of the 
input are called normal in the optimal control literature. It is well known that 
time-optimal controllers of normal systems have no singular conditions: condi- 
tions where the optimal input is undetermined for a finite time interval [6]. In 
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fact, according to the Pontryagin’s Maximum Principle [13], for a normal linear 
system, the time-optimal control exists, is unique, and is piecewise constant that 
taking values on the vertices of the feasible input set. Moreover, the optimal con- 
trol has a finite number of switchings if the dynamic matrix A has purely real 
eigenvalues. These results will be crucial in establishing the well-posedness of 
our models, and the termination of the following controller synthesis procedure. 

Controller Synthesis Methodology 

1. Apply Maximum Principle to obtain the saddle solution of optimal u*,d*. 

2. Construct a hybrid system using the switching logic of optimal u*,d* . 

3. Perform reachability computations on the constructed hybrid system. 

4. Compute the least restrictive controller. 

In the next sections, we describe in detail each step of the above procedure. 



3 Differential Games and the Maximum Principle 



In this section, we apply results from differential game theory [2,10] to formu- 
late the optimal control problem for our controller synthesis methodology. The 
Hamiltonian for the system (10), is given by H{x,p, u, d) = p^ Ax+p^ Bu+p^ Ed. 
The Hamiltonian satisfies the state and co-state differential equations 



_dH , _ dH^ 

dp' ^ dx 



(12) 



Consider the target set G = {a; G M” ] h{x) < 0}. By setting p(a;,0) = ^{x), 
then for every x G dG, p{x, 0) is the outward pointing normal to dG at x. With 
this initial condition, the co-state is completely specified by 

dh 

P(a^,0) = — (a;), p{x,f) = -A^ p{x,f). (13) 

Since the goal of the controller is to avoid G, the controller tries to maximize the 
Hamiltonian, while the disturbance tries to minimize it. In this case, the Isaacs 
condition [2], namely 



maxminiL(a;, », u, d) = minmaxiL(a;, », u, d), (14) 

«e(7 d&D deD ueu 



is satisfied since the Hamiltonian is separable, i.e. H{x,p,u,d) = H\{x,p,u) + 
H 2 {x,p, d). Satisfaction of the Isaacs condition implies that there exists a saddle 
solution of optimal controls and disturbances {u* ,d*) such that 



H{x,p,u,d*) < H{x,p,u*,d*) < H{x,p,u*,d). 



The saddle solution of optimal controls and disturbances u* , d* satisfies the well- 
known Maximum Principle [13] 



u*(xo,t) G argmaxueu p(xo,t)^dIu 
d*(xo,t) G argmindeDp(xo,t)^Ed. 



(15) 
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Equation (15) only constrains the optimal control and disturbance to lie in sets. 
We will soon see that under the normality condition, these sets are singletons, 
i.t. the optimal control and disturbance are unique. Starting from an initial 
xo G dG, the input u*{xq, •) is the best the controller can do to avoid G regard- 
less of the actions of the disturbance, while d*{xo, •) is the best the disturbance 
can do to drive the state towards G. These controls and disturbances are gen- 
erally open-loop (as opposed to feedback) policies and are so-called “bang-bang 
controls” since they switch among the vertices of the set of admissible controls 
and disturbances. Notice that due to the separability of the Hamiltonian, the 
problem of computing a saddle solution to the dynamic game reduces to solving 
two linear optimal control synthesis problems. 

Propositions 1 and 2 are fundamental for establishing the well-posedness of 
our controller synthesis methodology. The proofs are due to Pontryagin [13] and 
can be found in many optimal control texts, such as [6]. 

Proposition 1 (Nonsingular Optimal Control and Disturbance). If the 

linear system (10) is normal with respect to both the control and disturbance, 
then for any xq € dG, the optimal control u*(xo,-) and disturbance d*{xo,-) are 
unique and piece-wise constant taking values on the vertices ofU,D. 



Proposition 2 (Finite Switchings of Optimal Control). If the linear sys- 
tem (10) is normal and A has purely real eigenvalues, then there is a uniform 
upper bound, independent of xq on the number of switchings of the optimal con- 
trol u*(xo,‘), and disturbance d*{xo,-). 



4 Construction of Hybrid System 

The switching policy of the optimal control and disturbance can be naturally 
abstracted as a hybrid system. 

Definition 1 (Hybrid Systems). A hybrid system is a tuple H= {X, F, Inv, R) 
where 

— X = Xo X is the state space with Xo = {qo, ■ ■ ■ , qk-i}, 

— F : Xo X R™ ^ R™ assigns to each discrete location q G Xo a differential 
equation x = F{q,x), 

— Inv : Xo ^ assigns to each discrete location an invariant set Inv{q) C 
R"*, and 

— RCXxX is a relation capturing the discrete transitions . 

The elements of Xo are the discrete states whereas x G R™ is the continuous 
state. Hybrid systems are typically represented as graphs with vertices Xo, and 
edges E defined by 



E = {{q, q') G Xo x Xo \ {q, x, q' , x') G R for some x, x' G R™}. 
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With each edge e = {q, q') G E we associate a guard set defined as 

Guard{e) = {a; € Inv{q) \ {q,x,q',x') G R for some x' G M"*} 
and the set valued reset map 

Reset{e,x) = {x' G Inv(q') \ {q,x,q' ,x') G R}. 



Due to switched nature of the optimal control and disturbance, in this paper, 
it will suffice to assume that for all e G E, Reset{e, x) = x. Therefore, all 
reset maps will be the identity map. Furthermore, we do not require the explicit 
specification of any initial states for our hybrid system. 

The solution of the dynamic game played between the control and the dis- 
turbance d can be naturally encoded by a hybrid system. The optimal controls 
and disturbances always lie on the vertices of the admissible set of controls and 
disturbances U and D which are n„ and Ud dimensional rectangles. Thus, there 
are 2”“ • 2”*^ possible vector fields associated with the optimal controls and dis- 
turbances. We can therefore construct a hybrid system with 2”“ • 2"*^ discrete 
states, one for each possible control/disturbance pair. 

We naturally encode the discrete states as a string of boolean numbers of 
length Uu + nd- The first elements encode the value that the t-th component 
of the optimal control. Similarly the last Ud components encode the value of the 
optimal disturbance. We adopt the convention that 1 stands for the upper bound 
(u* = Ui or d* = Di), and 0 stands for the lower bound (u* = V_^ or d* = D_i). 
For example, in a system with two controls and one disturbance, the discrete 
state (0, 0, 1) stands for the case where u\ = U_i, U2 = U_2, and d^ = Di. It is 
therefore clear that the number of discrete states is 2”“+”'', since X]j contains all 
such boolean strings. According to which is notationally most convenient in the 
context, we will refer to discrete state k as either qk or the boolean string that 
represents k in binary. That is, for the example above we may refer to discrete 
state 5 as either <75 or (1,0, 1). 

Since the optimal control depends on the co-state p, the continuous state 
associated with the hybrid system is actually {x,p)'^ G The vector field 

with each discrete state qj then 

(^) = (0 -A^) + + 



where Uq^ G and dq^ G M"'' are the constant controls and disturbances 
associated with discrete state qj. 

Let (si , . . . , 1 1 , . . . , ) G Xu where all the Si and ti are either zero or 

one. Consider the formulas 



f > 0 if s = 1 

< 0 if s = 0 

f p’^{—Ay'^P^ei < 0 if s = 1 
> Oif s = 0, 



(17) 



(18) 
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where bi and are the columns of B and E respectively, and /3i(-),£i(-) are the 
relative degrees that are now defined. 

Definition 2 (Relative Degree). The relative degrees of the i-th input and 
disturbance are functions Pi,Si : M” — > Z defined by: 

r 0 if yf 0 

1 if p^bt = 0 A p'^{-A)b^ yf 0 
A(p) = < . 

i 3 if = ^3\p^{-Ayh y 0 

r 0 if y 0 

1 if p^Ci = 0 A p^{-A)ei y 0 
£*(P) = \ : 

i J if Afe=oP^(-^)''G = 0 Ap^(-A)^ei y 0. 

The invariant set associated with discrete state (si, . . . , ri, ■ ■ ■ , tna) is simply 

riu rid 

/nu((si,...,s„„,ti,...,t„J) = /\/“(si)A /\lf(tj). (21) 

^=l j=i 

In other words, the optimal control and disturbance remain the same as long as 
the signs of all components oip^ B and p^ E do not change. Proposition 1 ensures 
that components of p"^ B and p^ E cannot be zero for nontrivial intervals of time, 
and, furthermore, if some component of p"^ B or p^ E is momentarily zero, the 
optimal control and disturbance can be uniquely determined by looking at the 
first nonzero Lie derivative. 

Since, in general, the optimal policy can jump from any control/disturbance 
pair to any other control/disturbance pair, the edge relation E is all of Xu x Xn,. 
Consider discrete states (s( , . . . , , . . . , ) and (s^ ■ • ■ , s A > > ■ ■ ■ > 

let J„ be the set of indices z in {1, . . . , n„} such that s\ ^ sj. Thus contains 
the indices of all control components that switch optimal policy. Similarly define 
Jd- The guard that enables the transition e from (s}, . . . , , t}, . . . , A^) to 

{s\, ■ ■ ■ given by 

Guard{e) = f\ If {si) A f\ lf{t-). (22) 

where s denotes the boolean complement of s. 

Notice that for each discrete state, the invariant and the guard depend only 
on the co-state p. Therefore, there are formulas Inv^ : M” ^ {true, false} for 
j G {0, . . . , 2”“+"'' — 1} such that 

Inv{qj) = {{x,p)'^ G | InVj(p)} (23) 

The formulas Inv^ will be used for notational convenience in the reach set compu- 
tation of the next section. This concludes the specification of the optimal control 



(19) 



(20) 
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Fig. 1. Natural encoding of game solution as a hybrid system 



policy as a hybrid system. Figure 1 shows a block diagram of a hybrid system 
constructed out of a differential game between one control and one disturbance. 

From Propositions 1 and 2 it is straightforward to show that the hybrid 
system we construct is also well defined in the following sense. 

Proposition 3 (Properties of Hybrid System). The hybrid system con- 
structed above is nonblocking, deterministic, and non-Zeno. 

The problem of computing the maximal controlled invariant set W has thus 
been transformed to the problem of computing all states of the hybrid system 
constructed above that the x component of the continuous state can reach G. 
This reachability computation is the goal of the next section. 

5 Reachability Computation 

For the vector field associated with each discrete state qj , we define the pre- 
decessor operator Pre^- : 2* " — > 2* " . Suppose a set it' C is defined by 
K = G | P{x,p)}. Then Prej(iF) is defined by 

Prej(iF) = G I : P{y,q)At>0A q = p 

A y = e*^x + (/p e^^~‘^'>^ds){Buq^ + Edq^) (24) 
AVs:0<s<t=l> Invj(e“^^^p)}. 

An immediate corollary of the main theorem of [9], which is based on the 
results in [7,8], is the following: 

Proposition 4. Consider a semialgebraic set K C K" and a dynamic system 
X = Ax-\-b where A G b G Q". If A is nilpotent or diagonalizable with real 

rational eigenvalues, then computing the states that can reach K is decidable. 
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Proof. Suppose the K is defined by if = {a; G M” | P{x)}. By defining 
(j){x,t) = + f ds, 

Jo 

we have that the set of states that can reach K is given by {x G M” | 3y3t : 
P{y) A t > 0 A y = (j){x, t)}. In order to prove the result, we must show that for 
each condition on A above, (f>{x, f) can be converted to an equivalent formula 
in (K, <, +, •, 0, 1), which is decidable. If A is nilpotent, then each entry of 
is polynomial in t. Therefore each entry of 4>(x, t) is polynomial in t and hence 
definable in (R, <, +, •, 0, 1). If ^ is diagonalizable with real eigenvalues then each 
entry of is a linear combination of the functions with Xi an eigenvalue 
of A. Since the entries of are linear combinations of after integration 

the entries of 4>{x,t) are linear combinations of If A G Q, then by the 

procedure outlined in [9], may be converted into an equivalent formula in 

(R,<,+,-,0,l). □ 

An immediate result of Proposition 4 is that the computation of Prej{K) 
is decidable for each discrete state qj if AT is a semialgebraic set, and A is 
either nilpotent, or is diagonalizable with real eigenvalues. Notice that if K is 
semialgebraic, then so is Prej{K). 

Now, our goal is to compute all the states of the dynamical game (10) for 
which the disturbance can drive the state into reach the target set G regardless 
of the input. In fact, it is only necessary to compute the states that for which 
the disturbance can drive the state to the “Usable Part” of G: 

Definition 3 (Usable Part). The Usable Part (UP) of the target set G is the 
subset of dG for which the disturbance can instantaneously drive the state into 
G regardless of the control action. Thus UP for the dynamic game (10) and the 
target set (11) is given by: 

UP=iyX&dG\\Ju&U3d&D (^?f^Y{Ax + Bu + Ed)<Q^. (25) 

Since h{x) is a polynomial, then the defining formula (25) for UP is definable 
in the theory of the reals (R, <,+,-, 0, 1) which is known to admit quantifier 
elimination and be decidable [14]. Therefore computing UP is decidable. Since 
the hybrid system has the identity as its reset map, any trajectory that enters 
G if and only if is passes through UP. Now, we need to convert our reachability 
specification of the linear system (10) into a specification for our abstracted 
hybrid system. To this end, we define the set 

UP= {(x,p) gR2" I XGUP,P= (26) 

Define Pre : 2* ^2* such that for a given K C R^”, Pre(AT) is the set of all 

states of the hybrid system that can reach K. It is easily seen that 



IU° = {a; G R” I : {x,p) G Pre(UP) V a; G G} 



(27) 
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Therefore, the computation of W is decidable if and only if the computation of 
Pre(UP) is decidable. 

We now turn to computing Pre(UP). In general, different states on UP may 
r^uire different optimal control and disturbance values. We therefore partition 
UP into a disjoint union of subs^ according to the optimal controls and dis- 
turbances. That is, we partition UP = UgeQ >^91 where Q C Xd and the set Sq 

contains those state of UP for which the optimal control and disturbance of the 
are represented by discrete state q of the hybrid system. Since there are only 
2 "u+"d possible optimal controls and disturbances, the partition is finite. Using 
similar quantifier elimination arguments, it is straightforward to show that the 
computation of this partition is decidable. Since we have that 

Pre(UP) = Pre (u^^Q Sq) = [jq^g Pre(^,), (28) 

we can concentrate of computing Pre(5g) for a given q G Q. We know that the 
initial optimal control and disturbance is equal for all initial conditions in Sq, 
therefore Sq is contained within the same discrete state. 

Theorem 2 (Computation of Maximal Controlled Invariant Set). Con- 
sider a dynamic game x = Ax-\-Bu-\-Ed with controls u G U C M”“, disturbances 
d G D C and target set G C M" given by G = {x gK^ \ h{x) < 0}. Suppose 
the system and target set satisfy the following properties: 

1. A G B G E G 

2. For each column bi of B, the pair (A,bi) is completely controllable, 

3. For each column Ci of E, the pair {A, Ci) is completely controllable, 

4- The feasible sets of controls U and disturbances D are compact rectangles 
with rational vertices, 

5. h G Q[xi,X 2 , ■■■,Xn] and fy(a:) yf 0 when h{x) = 0. 

If A is nilpotent or diagonalizable with real rational eigenvalues, then the com- 
putation of the maximal controlled invariant set W is decidable. 

Proof. Due to our partition in equation (28), we have 

1U“ = {x G M" I : (x,p) G Pre(UP)}U G (29) 

= IJja; G K” I 3p: (a;,p) G Pre(S',)}UG (30) 

q&Q 

where each of the above steps is decidable. Thus it suffices to show that for a 
given q G Q computing Pre(S'q) is decidable. 

Due to Proposition 3, we need not worry about any pathologies in the Pre 
computation. Since, in each discrete state, the optimal input and disturbance are 
constant we apply Proposition 4 to decidably compute the set of states that can 
reach Sq for that particular combination of control/disturbance pair. However, 
the optimal control or disturbance may change and a discrete transition may be 
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taken. The predecessor operator of the discrete jumps is trivial since the reset 
map of our jumps is the identity map. 

Now, if the matrix A has real eigenvalues, then due to Proposition 2 after a 
finite number of switchings, uniformly in G Sg, there are no more switches 

and we can use Proposition 4 one last time. Therefore, the algorithm terminates 
after a finite number of steps. □ 



6 Least Restrictive Controller 

Our goal in this section is to compute the least restrictive controller that renders 
the maximum controlled invariant set W invariant. The result of the previous 
section is that W is definable in (K, <,+,-, 0, 1) which is decidable [14]. Since 
(R, <,+,-, 0, 1) admits quantifier elimination, we may compute a quantifier-free 
formula such that W = {x G M" | ipix)}. The quantifier elimination that is 
required in this procedure can be done by the computer logic software systems 
Redlog [5] or Qepcad [4] . The defining formula of the set W may be converted 
to the so-called disjunctive normal form to yield: 

VP = {a: e R" I V ^=1 (A^i a. mj, o) } (31) 

where G Q[xi,... ,x„] and G {<,<,<,>,=, yf}. 

Since the least restrictive controller specifies a control action only on the 
boundary of W, our first task is to compute the boundary of W, dW. We will 
need the following lemma from [17]. 

Lemma 1. IfWC R” is definable in a decidable theory, then so is the closure 
W, the interior W°, and the boundary dW . 

Proof. For a set W = {a; G R” | if{x)}, the sets W and W° are given by 

VP = {a; G R” I V(yi,... , y„) V(zi, . . . , z„) : [A(Li2/» < Xi < Zi ^ (32) 

3(wi, . . . ,Wn) : <Wi < ZiA i>{w)]} 

VP° = {a: G R" I 3(yi,... , j/„) 3(zi, . . . ,z„) : [A(bi?/i < x^ < Zi A (33) 
V(wi, . . . ,Wn) : A(Li?/i <Wi < Zi^ 

where we use the shorthand notation (a /3) = {-^a V (3). The expressions (32) 
and (33) are simply the definitions of closure and interior in the usual topology 
of R". Let the defining formulas for VP, VP° be ip, ip° respectively. Then the 
defining formula for 9VP is simply dip = ip A {~'ip°). Clearly if ip is defined in a 
theory which admits quantifier elimination, then so are ip, ip°, and dip. □ 

From the Lemma 1 we have that dW may be defined by a quantifier-free 
formula 



91F = {a; G R" I dip(x)}. 



(34) 
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Since the least restrictive controller only specifies a control action on dW , then 
g : K” ^2^ must be of the form 

g{x) = {u€U \ d'ip{x) 4>{x,u)} (35) 

where (j){x, u) is a formula to be described below. 

Denote W = Uj=i(rifc=i consider the least restrictive controller 

for a single polynomial constraint Wj^ = {x G M" | nij^ 0}. For this 

polynomial constraint, we define the formula 

{x, u) = (a;) = 0) ^ Vd G D : {Ax + Bu + Ed) 0^ . (36) 

Using equation (36), it is direct to see that the least restrictive controller that 
renders Wj^ invariant is given by gj^{x) = {«. G C/ | 4>j^{x,u)}. This least re- 
strictive controller is simply a re-writing of equation (9) in terms of a decidable 
formula. Now, the least restrictive controller for W = Ut=i(nfl^ must be 
satisfy each of the of the simpler constraints, and hence is given by the following. 



Theorem 3 (Least Restrictive Controller). For the differential game x = 
Ax + Bu 4- Ed, the least restrictive controller g : K" ^ 2^ that renders the set 
LF = {x G M" I V^=i(Afci fjk(x) Txijk 0)} invariant is given by 

g{x) = G [/ I dip{x) =4 VjAi (A^i (x, u)) } , (37) 

where dip is the defining formula of dW and (p>jffx,u) is given by equation (36). 
IfW is definable in a decidable theory, then so is g{x). 

Therefore, Theorems 2 and 3 collectively result in Theorem 1. 

7 Conclusions 

In this paper we have shown that controller synthesis for classes of linear systems 
with polynomial reachability specifications is decidable. In further research, we 
will extend the target set G to a semialgebraic set, investigate conditions for 
semi-decidability in the absence of the normality condition, and extend the con- 
tinuous decidability results to semidecidability results for classes of linear hy- 
brid systems. In the case of purely imaginary eigenvalues, the problem becomes 
quickly undecidable unless one remains in a compact region of the state space. 
The observation along with the results of this paper have a clear and natural 
connection with o-minimal theories of the reals [7,17], which will explored in 
future research. 
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Abstract. The main purpose of this paper is to introduce a new frame- 
work for a global, geometric study of hybrid systems, and demonstrate 
its usefulness through its application to the analysis of the Zeno phe- 
nomenon and stability of hybrid equilibria. 



1 Introduction 

In this paper we present a unifying approach for treatment of hybrid systems. We 
define the notions of the hybrid manifold (or hybrifold) and hybrid flow, which 
enable us to study the hybrid system “in one piece”, that is, as a single, generally 
non-smooth dynamical system. 

Having established a reasonable framework for the geometric study of hybrid 
systems as dynamical systems, we focus particularly on the Zeno phenomenon, 
which does not occur in smooth dynamical systems. We study its causes, ways 
of removing it from the system, and classify it topologically in dimension two. 

The last part of the paper deals with stability of isolated hybrid equilibria. We 
prove a theorem which explains, among others, examples in which a stable hybrid 
equilibrium is composed of unstable classical equilibria. Proofs of all statements 
in the paper can be found in [SJSL]. 

2 Preliminaries 

2.1 Definitions and Examples 

Definition 1. An n-dimensional hybrid system is a 6-tuple H = {Q,E,V,X, 
G,TV), where: 

— Q = {1, . . . ,k} is the collection of (discrete) states of H, where k>l is an 
integer; 

— E C Q X Q is the collection of edges; 

* This work was supported by the NASA grant NAG-2-1039, the Swedish Foundation 
for International Cooperation in Research and Higher Education, Telefonaktiebo- 
laget L.M. Ericsson, ONR under N00014-97-1-0946, DARPA under F33615-98-C- 
3614, and ARO under DAAH04-96-0341. 
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R(2,1) 

Di 



G{2, IB 



G(i;2)- 



D2 



R(1,2) 

Fig. 1. The water tank example. 



— T> = {Di : i G Q} is the collection o/ domains^ ofH, where Di C {z} x M” 
for all i G Q; 

— X = {Xi : i G Q} is the collection of vector fields such that Xi is Lipschitz 
on Di for all i G Q; we denote the local flow of Xi by {4>l}. 

— G = {G{e) : e G E} is the collection 0 / guards, where for each e = (i,j) G E, 
G{e) C A; 

— TZ = {Re : e G E} is the collection 0 / resets, where for each e = (i,j) G E, 
Re is a relation between elements of G{e) and elements of Dj, i.e. Re C 
G(e) X A. 

Remark. If a reset relation Re is actually a map G(e) ^ Dj, with e = (z, j) G E, 
instead of {x, y) G Re we write y = Re{x). Observe that domains Di lie in distinct 
copies of M". However, we will sometimes abuse the notation and consider the 
domains as subsets of a single copy of M”. We also set D = UieQ 
call this set the total domain of H, and G = Uee£;^(®)’ ^ ~ UeeE ^e{G{e)), 
g = {~^:eGE}, n={Re{G{e)) : e G E}. 

Given H, the basic idea is that starting from a point in some domain Di we flow 
according to Xi until (and if) we reach some guard G(i,j), then switch via the 
reset R(ij), continue flowing in Dj according to Xj and so on. 

Example 1 (Water Tank WT). Here n = 2, k = 2, E = {(1, 2), (2, 1)}, D\ = 
{l}xG, D 2 = {2}xG, where C = [h,(X))x[l 2 , 00 ), Xi = {w—vi, —V 2 )'^ , X 2 = 
{-vi,w - U 2 )^, G(l,2) = {(l,xi,a: 2 ) G Di : X 2 = h}, G(2, 1) = {(2,a;i,a;2) G 

D2: xi = h}, and i?(i_2)(l,a;i,Z2) = (2,xi,,l2), i?(2,i)(2, Zi, 0:2) = (l,^i,a:2). 

The interpretation is as follows (cf. Fig. 1). For i G Q, Xi denotes the volume 
of water in tank i, Vi is the constant rate of flow of water out of tank z, and li 
is the desired volume of water in tank i. The constant rate of water flow into 
the system, dedicated exclusively to one tank at a time, is denoted by w. The 
control task is to keep the water volume above h and I 2 (assuming the initial 
volumes are above R and I 2 respectively) by a strategy that switches the inflow 
to the first tank whenever x\ = l\ and to the second tank whenever X 2 = 12 - 



Example 2 (Bouncing Ball BB). 

This is a simplified model of an elastic ball that is bouncing and losing a 
fraction of its energy with each bounce. We denote by x\ its altitude and by 



^ In the literature also known as “invariants” . 
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-R(i.i) 

G(l, IL, 



Di 



Fig. 2. Bouncing ball. 



X2 its vertical speed. Here n = 2, k = I, E = {(1,1)}, Di = {{xi,X2) ■ x\ > 
0}, Xi{xi,X2) = (x2,-g)'^, G(l,l) = 1(0, a;2) : X2 < 0}, i?(i.i)(0, X 2 ) = 

(0, —CX2), where g is the acceleration due to gravity and 0 < c < 1 (cf. Fig. 2). 

Example 3 (Bouncing m-Ball BB{m)). 

The only difference between this and the previous example is that we have 
m different domains in which the ball can bounce and after each bounce the 
ball switches to the next domain in a cyclic order. That is, n = 2, fc = m > 1, 
if = 1(1, 2), (2, 3), ... , (m— 1, m), (m, 1)}, and for alH € Q, Di = {z|x{(xi, 0 : 2 ) : 
xi > 0}, G{i,i+l) = (zjx {( 0 , 0 : 2 ) : X 2 < 0}, i?(i,*+i)(z, 0, 0 : 2 ) = (z+1, 0, - 00 : 2 ), 
where we conveniently identify m + 1 := 1. Note that here the domains are just 
different copies of the closed right half-plane in 

Example 4 (Ball Bouncing on an N-step Staircase BBS{N)). 

Here a ball is bouncing on an iV-step staircase. Assume that step z = 1, . . . , 
has width Wi > 0 and height hi > 0, and define Wm = and hm = 

Y(aLi hi- Assume also that the ball loses a proportional amount of its vertical 
velocity ( 0 : 2 ) with each bounce and that the ball has constant horizontal speed 
( 0 : 3 ). Denote by o;i its vertical position. Then we have: Q = {1, . . . , iV -|- 1}, 
E = {(z, z) : 1 < z < N -I- 1} U {(1, 2), . . . , {N, N + 1)}, and for 1 < z < 
A^ -I- 1: A = {*} X [/zi,oo) X (— 00 , 0 ] x (— oo,zij], G{i,i) = {(a;i, X 2 , X 3 ) G 
Di : xi = hi}, R(^^^i){i,xi,X 2 ,xz) = {i,xi,-cx 2 ,xz) and Xi{xi,X 2 ,X 3 ) = 
{x 2 , —g,v)'^ . Furthermore, for 1 < z < A^: G(z,z-|- 1) = {{xi,X 2 ,x^) G Di : X 3 = 
Wi}, i?(i,i+i)(z, x) = (z -I- l,x). For more details see [JLSMj. 

Example 5 (Two Saddles S2{\)). 

Here n = 2, k = 2, X > 0, E = {(1,2), (2, 1)}, the domains are two copies 
of the square S = [—1,1] x [—1,1], i.e. for i G Q, Di = {z| x S, Xi{x\,X 2 ) = 
{Xxi,—X 2 )'^, X 2 {x\,X 2 ) = {—x\,Xx 2 )^, G{1,2) = union of the vertical sides 

of Di, G(2, 1) = union of the horizontal sides of D 2 , R(^ij){i,x) = (j,x), for all 
(z,j) G E. 

Example 6 (Flow on the 2-torus 

We have a > 0, n = 2, k = 2, E = {(1, 2), (2, 1)}, Di = {z| x K, where 
K = [0,1] X [0,1] is the unit square, Ai = A 2 = (l,o;)^ are constant vec- 
tor fields, G{i,i) = {zj x S'upper, G{i,j) = {z| x S'j.jgpt, i?p,q(z, a:, 1 ) = 
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{i,x,0) and R{ij){i, l,y) = {j,0,y), where i,j = 1,2, i ^ j, S'upper = [0, l]x{l} 
and = {1} X [0,1) denote the (closed) upper and (half-closed) right side 

of K. Note that x iS^pper) x *5^iower ^ “^right) 

{j} X S'jgf^, with the obvious meaning of -S'jQ^gj. and S'lgft- 

If we proceed as is usually done in geometry and identify {t} x S'upper with 
{*} X 5'lower via and {i} x Slight with {j} x Sjgft via R(ij) (where i,j = 1, 2, 
i yf j), we obtain the standard 2-torus with a smooth flow with slope a on it. This 
is a baby-version of a construction we will later apply to more general hybrid 
systems. 

Keeping in mind the examples above, we formally define the notion of an exe- 
cution of a hybrid system. 

Definition 2. A (forward) hybrid time trajectory is a sequence (finite or infi- 
nite) T = {/j}S^Q of intervals such that Ij = [ry, rj] for all j > 0 if the sequence 
is infinite; if N is finite, then Ij = [rj, rj] for allO < j < IV — 1 and In is either 
of the form [tat,t)^] or [tatjT)^). The sequences tj and rj satisfy: tj < rj = Tj+i, 
for all j. 

One thinks of tj’s as time instants when discrete transitions (or switches) from 
one domain to another take place. If t is a hybrid time trajectory, we will call N 
its size and denote it by N{t). Also, we use (r) to denote the set {0, . . . , N{t)} 
if N(t) is finite, and {0, 1,2,...} if N(t) is infinite. 

We will say that r is a prefix of an execution r' = if IV < IV' (where 

the inequality is taken in the extened real number system), and for 0 < j < N, we 
have Ij = I'j] furthermore, if r has finite size, then we must also have /at C I'n- 

Definition 3. An execution (or forward execution) of a hybrid system H is a 
triple X = (T,q,x), where t is a hybrid time trajectory, q : (t) ^ Q is a map, 
and X = {xj : j G (r)} is a collection of maps such that Xj : Ij — > Dg(^j) 
and for all t G Ij, Xj{f) = Xg;j;(xj(f)). Furthermore, for all j G (r), we have 
+ 1)) G E, Xj(T)) G G{q(j),q{j + 1)), and (xj(r'), Xj+i(tj+i)) € 
E(q{j),q{j+'^))- 

For an execution y = (T,q,x), denote by Too(x) its execution time: Taofx) = 
-Tj)= \imj^N(r) r'j - To. 

Definition 4. An execution y is called: 

— infinite, if N{t) = oo or Too(x) = oo; 
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— a Zeno execution if N{t) = oo and Too{x) < oo; 

— maximal if it is not a strict prefix of any other execution of H. 

The last statement means that there exists no other execution x' = 

such that r is a strict prefix of r' and x = x' on t (in the sense that Xj = x' on 

Ij for all j G (t)). 

Note that in Examples 1 {WT), 2 {BB) and 3 {BB{m)) every execution is 
Zeno. The same can be shown for Examples 4 (BBS{N)) if 0 < c < 1 and 5 
(S'2(A)) if 0 < A < 1. On the other hand, every execution in Example 6 (T^(a)) 
is infinite with infinite execution time. 

We say that an execution y = (r, g, x) starts at a point p € D ii p = xo(to) 
and To = 0. It passes through p ii p = Xj(t) for some j G (t), t G Ij, t > tq. 

Given p G D, it is not difficult to see that there are many ways in which a 
hybrid system can accept several executions starting from or passing through p. 
For instance, this happens if at least one of the resets is a relation which is not 
a function. 

Definition 5. A hybrid system is called deterministic if for every p G D there 
exists at most one maximal execution starting from p. It is called non-blocking 
if for every p G D there is at least one infinite execution starting from p. 

Necessary and sufficient conditions for a hybrid system to be deterministic and 
non-blocking can be found in [LJSE]. Roughly speaking, resets have to be func- 
tions, guards have to be mutually disjoint and whenever a continuous trajectory 
of one of the vector fields in X is about to exit the domain in which it lies, it 
has to hit a guard. 

2.2 Standing Assumptions 

From now on we will assume that every hybrid system H = {Q,E,T>, X ,Q ,IZ) 
in this paper satisfies the following assumptions. 

(Al) H is deterministic and non-blocking f 

This means that every point in D is the starting point of a unique infinite 
(and therefore maximal) execution of H. 

(A2) Each domain Di is a contractible n-dimensional smooth submanifold of 
K", with piecewise smooth boundary. No two smooth components of the boundary 
meet at a zero angle. 

The non-zero angle requirement eliminates, for instance, cusps in dimension 
two, but does not eliminate “corners”. Thus for domains of a hybrid system we 
allow disks, half-spaces, rectangles, etc. 

(A3) Each guard is a piecewise smooth {n — 1)- dimensional submanifold 
of the boundary of the corresponding domain. The boundary of each guard is 
piecewise smooth (or possibly empty). 

^ These assumptions can be relaxed. However, to simplify the exposition and avoid 
some nonessential technical difficulties in the subsequent construction, we keep them 
in the present form. 
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Fig. 4. Pi is of Type (Roman) i (1 < z < 4). 



(A4) Each reset is a piecewise smooth homeomorphism onto its image. The 
image of every reset lies in the boundary of the corresponding domain. 

(A5) Any sets inQUTZ (i.e. closures of guards and images of resets) can 
intersect only along their boundaries. Furthemore, if p G G U R, then p can be 
of only one of the following four types (cf. Fig. J^): 

Type I ; p e int G U int R; 

Type II : p G dGVddR and there exists exactly one set S G GGiTZ which contains 
p; _ _ 

Type III ; p G dG U dR and there exist sets Si,. .., Si G G GiTZ (I > 2) such 

that p G dSi n . . . n dSi and some neighborhood of p in S\ \J ... Si is 

homeomorphic to 

Type IV ; p G dG U dR and there exist sets S\,. . . , Si G G 'S'R (I > 2) such 

that p G dSi n . . . n dSi and some neighborhood of p in S\ ... Si is 

homeomorphic to 

Assumption (A5) ensures that intersections of guards and images of resets (that 
is, their closures) are sufficiantly nice. This in particular means that the config- 
uration around ps in Fig. 4 is not allowed. 

(A6) For all e = (i,j) G E, Xi points outside Di along G(e), and Xj is 
points inside Dj along im R^ . 

This means that if p € G{i,j), q = R(^ij){p), then there exists e > 0 such 
that (pStip) € mtDi and fUq) G intDj, for all 0 < t < e, where int denotes the 
interior of a set. In particular, we have that Xi is transverse to the smooth part 
of G(e) and Xj is transverse to the smooth part of imi?e, the image of the map 
Re- 

LA7) Each reset map Re extends to a map Re defined on a neighborhood of 

G{e) (the closure ofG{e)) in Di such that Re is a piecewise smooth homeomor- 
phism onto its image, which, in turn, is a neighborhood of im Re in Dj . Each 
vector field Xi can be smoothly extended to a neighborhood of Di in {z} x M”. 

The last one is a fairly technical assumption the need for which will be- 
come apparent later. Note that all the examples provided above satisfy this (as 
well as all other) assumptions. For instance, in Example 2 (BB), we can take 
R(l^l){xi,X2) = (XI,-CX2). 
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Definition 6. A hybrid system which satisfies assumptions (Al) - (A7) will be 
called regular. 

Given H, define a map ■. Qq ^ D, (where f7o C M x Z? will be specified later) 
as follows. Let p G D be arbitrary. Because of (Al), there exists a unique infinite 
execution x{p) = starting at p. For any 0 < t < Too(x(p)) there exist 

a unique j & Q such that t G [rj,Tj). Then define <P^{t,p) = Xj{t). To define 
<P^{t,p) for negative t, set <P^{t,p) = {—t,p), where H' is the reverse hybrid 

system {Q' , E' ,V , X' ,Q' ,TZ') defined by: Q' = Q, V = V, X' = —Xi, for all 
i G Q; (i,j) G E' if and only if (j,i) G E; and for every e = (i,j) G E', we have 
G'{e) = R(^j^i){G{j,i)) and R'^ = R~^ . 

It can easily be checked that H' satisfies (Al) - (A7) if H does. Now let Gq 
be the largest subset of M x D on which is defined. 

For instance, in Example 2, for any p yf 0, <P^^{t,p) ^ 0, as t — > Too(x(p)), 
where x(p) is the unique infinite execution starting at p. Note, however, that 
x(0) makes no time progress, i.e. Tj = 0 for all j > 0, but it involves infinitely 
many switches at the same, i.e. initial point, which happens to be fixed by the 
reset map. 

Theorem 1. (a) Gq contains a neighborhood of {0} x int D in M x D. 

(b) For all p G D, F^{0,p) = p. Furthermore, ^^(t, <?^(s,p)) = ‘F^{t + s,p), 
whenever both sides are defined. 

3 The Hybrid Manifold and Hybrid Flow 

The basic idea in construction of the hybrid manifold from a hybrid system 
is simple: “glue” the closure of each guard to the image of the corresponding 
extended reset via the extended reset map. Some relatively similar ideas appear 
in [GJ]. 

3.1 The Hybrifold 

Let H be a regular hybrid system. On D let ~ be the equivalence relation 
generated by p ~ Re{p), for all e G if and p G G(e). Gollapse each equivalence 
class to a point to obtain the quotient space Mh = D/ ~ . 

Definition 7. We call Mh the hybrid manifold or hybrifold o/H.^ 

Denote by tt the natural projection D — > Mh which assigns to each p its equiv- 
alence class p/ Put the quotient topology on Mh. Recall that this is the 
smallest topology that makes tt continuous, i.e. a set V C Mh is open if and 
only if TT~^{y) is open in D. 

Define the hybrid flow of H, Mh, by F^{t,TT{p)) = 7r^^(t,p). 

Here fl = {(t,7r(p)) : (t,p) G l7o}- In other words, orbits of are obtained by 

® The authors thank Renaud Dreyer for suggesting the term hybrifold. The term 
“manifold” will be justified by Theorem 2. 
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Fig. 5. Hybrifold 




Fig. 6. Hybrifold 




and an orbit of the hybrid flow for BB. 



projecting orbits of by tt. By the ^^-orbit of p we mean the collection of 
points <P^{t,p) for all possible t (i.e. all t such that (t,p) G l7o)- 

Let us run this construction on some of the examples listed above. 

Example 1 (WT continued). 

Without loss we assume that = ?2 = 0. To obtain Mwt we have to identify 
the xi-axis from D\ with the same axis from D 2 via ^( 1 , 2 ) £md similarly with 
the a; 2 -axis. 

It is not difficult to see that Mwt is homeomorphic to (see Fig. 5). 
However, Mwt has a singularity (or “corner”) at 0 = 7r(l,0,0), i.e. tt does not 
define a smooth structure on Mwt- Note that every execution starting at a: yf 0 
converges to 0 . 



Example 8 (BB continued). 

Here we have to identify the negative part with the positive part of the x^- 
axis. The resulting space Mbb is again homeomorphic to (see Fig. 6), but 
7T again does not define a smooth structure on it. As in the previous example, 
E^^{t,x) ^ 0 , as t ^ Tocixi^))? for aU x ^ 0 . 



Example 9 (BB{m) continued). 

For simplicity assume m = 2. It is not difficult to see that Mbb(2) is smooth 
(in the sense explained above) and diffeomorphic to However, the hybrid 
flow is not smooth. 



Example 10 (S2{\) continued). 

Ms 2 (\) is homeomorphic to the 2-sphere; it is not equipped with a smooth 
structure by tt. 
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Example 11 (T‘^{a) continued). 

We already observed that is the standard 2-torus and is a 

smooth linear flow on it. If a is rational, then every orbit is closed; if a is 
irrational, then every orbit is dense in T^. 

Theorem 2. (a) Mu defined above is a topological n-manifold with boundary. 

(b) Both Mu and its boundary are piecewise smooth. 

(c) The restriction ^ : int Z? — > 7r(int D) is a diffeomorphism. 

Recall that M is called a topological n-manifold with boundary if it is Hausdorff 
and every point in M has a neighborhood homeomorphic to either M” or the 
closed upper half-space M” = {(xi, . . . ,x„) : Xn > 0}. Points having the latter 
property are said to be on the boundary dM, which is a topological (n — 1)- 
manifold. 



3.2 The Hybrid Flow 

Let E := be the hybrid flow of H, as defined above. For each t G M and 
X G Mil, let M{t) = {y £ Mh : T{t,y) is deflned}, and J{x) = {s G M : 
E{s, x) is deflned}. Observe that if a; = 7r(p), then J{x) n [0, oo) = [0, Too(x(p))). 
Also, for t > 0, M{f) contains all points x = tt{p) such that Too(x(p))) > t. As 
usual, xip) denotes the unique execution of H starting at p. 

If M{t) is not empty, denote by Tt : M{t) — > Mh the time t map of T, 
deflned by Tfix) = E{t,x). Recall that a function (in particular, vector held) is 
said to be smooth on a closed set F if it is the restriction of a smooth function 
deflned on a neighborhood of F. Then we have the following theorem. 

Theorem 3. Suppose each vector field X in X is smooth (in addition to being 
globally Lipschitz). Then: 

(a) For each x G Mh the map t i-^- Ffix) is continuous and, if J{x) is not a 
single point, piecewise smooth on J{x). More precisely, it is smooth except 
at (at most) countably many points in J{x). Furthermore, each map Ft is 
injective. 

(b) Whenever both sides are defined: 

(c) There is an open and dense subset of fl on which F is smooth. 

4 cj-Limit Sets and the Zeno Phenomenon 

It has to be pointed out that Zeno executions do not arise in physical systems 
and are a consequence of modeling over-abstraction. Therefore, one wishes to 
avoid them. However, from a mathematical viewpoint, the Zeno phenomenon 
poses numerous interesting questions. In this section we show that, in short, the 
topological cause of Zenoness is a lack of smoothness in the hybrid flow and that 
the Zeno phenomenon can be removed by smoothing out the hybrifold and the 
hybrid flow on it. 
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Definition 8. A point y S Mh is called an uj-limit point of x G Mu if y = 
lim„ — too ^ ^ some %ncveas%ny sequence (fuf) m Ji^x^ such that t^ ^ 
Too(a^), as m ^ oo. The set of all ui-limit points of x is called the uj-limit set of 
X and is denoted by u>{x). 

By Too(x) we denote the execution time of the unique execution of H starting 
from p, where x = 7r(p); that is, Too{x) = Too(x(p))- K is easy to check that this 
is a well defined element of the extended real number system. In other words, 
w-limit points for x are accumulation points of the orbit of x. 

Suppose X € Mh and denote by Eao{x) the set of discrete transitions which 
occur infinitely many times in the execution starting from x. If E^o (a:) is empty, 
then the orbit of x eventually ends up in a single domain Di (that is, its image 
under tt in the hybrifold) in which case lv{x) C Tr{Di). This means that every 
point y G uj{x) is an accumulation point of the orbit of a single vector field, 
namely Xi. We will call such a point y, a pure co-limit point. 

If Eao{x) is nonempty, then every w-limit point for a; is a result of both 
the continuous and discrete (i.e. hybrid) dynamics of H and will accordingly be 
called a hybrid co-limit point of x. 

Theorem 4. For every x G Mh, co{x) is invariant with respect to the hybrid 
flow. That is, if y G oj{x), then T^{y) € co{x), for all t G J{y). 

4.1 Properties of Zeno Executions 

Definition 9. A point z G Mh is called a Zeno state for x if z G co{x) and 

Toc(x) < OO. 

We will also refer to points in 7t~^(z) as Zeno states in H. For example, the 
“origin” of Mwt (as well as Mbb and Mbb{ 2 )) is a Zeno state for every point. 
Moreover, for each x, co{x) contains only one Zeno state. We now show this is 
always the case. 

Theorem 5. If the execution starting from x G Mh is Zeno, then co{x) consists 
of exactly one Zeno state for x and u>{x) C riee£;oo(a:) 

Note than in all the Zeno examples above none of the flows involved in 
creating the Zeno state has an equilibrium at the Zeno state. The following 
lemma shows that this is not a coincidence. 

Lemma 1. A Zeno state is not a standard equilibrium (cf. Def. 12). More specif- 
ically, if z G Mh is a Zeno state, then for every p G tt~^{z), if p G Di, then 
Xflp) ^ 0. 



Example 12 (equilibrium + cusp = Zeno). 

Consider the following one-domain hybrid system: D = {{x,y) G : y > 
0, -f{y) <X< f{y)}, G = {{-f{y),y) :y>0}, R{-f{y),y) = {f{cy),cy), 

X (x, y) = (—X — y,x — y)'^ . Here 0 < c < 1, / : [0, oo) ^ [0, oo) is a smooth 
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function such that /(O) = 0 and for all y > 0, f{y) < In particular, /'(O) = 0, 
which means that D has a cusp at 0. It is not difficult to check that 0 is a Zeno 
state despite the fact that it is an equlibrium for X. This shows the importance 
of geometry of domains and assumption (A2). 



Theorem 6. Suppose H is a hybrid system such that its hybrid flow is 
smooth. (This in particular means that its hybrifold Mu is smooth.) Then H 
admits no Zeno executions or equivalently, there are no Zeno states in Mu- 

In general it may not be easy to check whether, given H, the hybrifold Mu 
is smooth. Even if it were, non-smoothness of the hybrid flow may cause Zeno 
(cf. BB(2)). However, the following result provides an easily verifiable criterion 
for smoothness of 

Theorem 7. Suppose that Mu is smooth and for every e = (i,j) G E, Xi 
and Xj are Re-related on G{e). That is, for every p G G{e): T Re{Xi{p)) = 
Xj{Re{p)). Then the hybrid flow is smooth. 



Example 13. Consider BB{2). Here we have: Xx{x\,X 2 ) = (x 2 ,—g)^ = X 2 , 
R(ijfli, xi,X2) = {j,xi, —CX2), where (z,j) = (1,2) or (2,1). It is easily seen 
that TR(^i 2 ){Xi) yf X 2 . Recall that the hybrid flow for BB(2) is not smooth. 



Example 1). It is not difficult to check that in case of T‘^{a), the condition from 
Theorem 7 is satisfied for every a > 0. Thus T‘^{oi) does not admit Zeno, as was 
already shown above. 



Corollary 1. //H is a hybrid system satifying condition from Theorem 1, then 

H accepts no Zeno executions. 

4.2 Removal of Zeno 

Suppose that H is a regular hybrid system and that 2 G Mu is a Zeno state. 

We have seen that Mu in a certain sense has a singularity at z. Consider the 

following ways of removing such singularities. 

Smoothing. Suppose that Mu can be equipped with a smooth structure which 
induces the same topology as the original one and denote the smoothed hybri- 
fold by (cf. Fig. 7). Note that Mu and are homeomorphic. 

It is not guaranteed that the hybrid flow will be smooth on 
If, however, is smooth with respect to the differentiable structure on 
then Theorem 6 implies that there are no Zeno states in 
We say that we have removed Zeno by smoothing. 
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Fig. 7. Smoothed water tank 



suspension of R{ 2 ,i) 



D2 






suspension of R(i, 2 ) 



Fig. 8. e-suspended water tank S'^Mwt- 

Hybrid suspension. ^ The basic idea is to “interpolate” executions between 
guards and images of corresponding resets, i.e. to make “instantaneous” 
discrete transitions given by reset maps “last” some time e. The constructions 
goes as f ollows . Let e > 0 be arbitrary and assume e = (i,j) G E. Instead 
of gluing G(e) to im Re via Re, first enlarge the domain Di by D\ = DiU 
(G(e) X [0,e]), and then identify (p,e) ~ Re{p), for every p G G(e). Denote 
the space obtained by this identification for all e G if by S'^Mn and by 
7T*^ the quotient (i.e. identification) map. On each G(e) x [0,e], consider the 
trivial “vertical” flow: (p,s,t) {p,s + t) {p G G(e), 0 < s < e, t G M). 
Denote by the flow on S'^Mn obtained by projecting via tt*^ this flow 

(for each e G E) as well as We will call S'^Mu the e-suspended hybrid 
manifold and the associated e-suspended hybrid flow (see Fig. 8). (This 

construction resembles the standard suspension of a map; cf. e.g. [PdM].) It 
is immediate by construction that for ever e > 0, accepts no Zeno-type 

executions. 



5 Conjugacy of Hybrid Systems and Classification of 
Zeno States in Dimension Two 

In this section we discuss the following question: when are two hybrid systems 
qualitatively the same? For that purpose we borrow the notion of conjugacy from 
the theory of dynamical systems. Roughly speaking, two dynamical systems are 
conjugate if their phase portraits look qualitatively (or topologically) the same. 
Similarly, two hybrid systems are conjugate if their hybrid flows are conjugate. 
We now make this more precise. 

^ We thank Morris W. Hirsch for suggesting this idea in a recent conversation. 
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Definition 10. Two hybrid systems Hi and H 2 are said to be topologically 
conjugate (denoted by Hi « H 2 ^ if there exists a homeomorphism h : Mhi — > 
Mh 2 which sends orbits of to orbits of . If Mhi and Mh 2 happen to 
be smooth manifolds of class C"" (r > 1) and h is a diffeomorphism, then Hi 
and H 2 are said to be -conjugate. 

As usual, by the orbit of a point x under a (local) flow {4>t} we mean the set 
of points 4>t{x) for all t for which 4>t{x) is defined. We usually think of ft- as a 
change of coordinates so that two hybrid systems are topologically conjugate if 
their hybrid flows are the same up to a continuous coordinate change. Note that 
conjugacy does not necessarily preserve the time parameter t. If it does, it is 
called equivalence. 

Example 15. WT is topologically conjugate to BB. This can be seen by suitably 
projecting Mwt and Mbb onto so that both and look like a spiral 
sink at the origin. For more details, see [SJSL] . We will see later that in dimension 
two this picture is typical. 

Example 16. T^(l) is not conjugate to T^(i/2). Even though the hybrifold for 
both hybrid systems is the same (the 2-torus), every orbit of T^(l) is closed, 
while every orbit of T^(-\/2) is dense in T^. 

Even though it is not possible to classify all hybrid systems up to conjugacy 
(this attempt fails even for smooth dynamical systems) , the next theorem shows 
that near a Zeno state, every 2-dimensional hybrid flow looks like near 0. 

Theorem 8. Let H &e a 2-dimensional hybrid system and suppose that z G Mu 
is a Zeno state. Then there is a neighborhood U of z in Mu and a neighborhood 
V of 0 in Mwt such that is topologically conjugate to 

6 Stability of Hybrid Equilibria 

Recall that if (ft is a local flow generated by a smooth vector held X on some 
set U (in M” or any manifold), then p S C/ is an equilibrium for X (equivalently: 
for (ft) if AT(p) = 0 (equivalently: if (ft{p) = P for all t G M.). In case of a hybrid 
system there is usually more than one vector held at play, and even in the case 
when there is only one, resets are involved in generating the hybrid dynamics. 
Taking this into account we define a hybrid equilibrium as follows. 

Definition 11. Let H. be a hybrid system. A point x G Mu is called an (hybrid) 
equilibrium for the hybrid flow ifE^(t,x) = x for all t G J{x). 

Equivalently, x G Mu is a hybrid equilibrium if the hybrid dynamics of H, con- 
sisting of reset maps and local flows of H, map tt~^{x) to itself. For example, any 
Zeno state is a hybrid equilibrium despite Lemma 1; however, hybrid dynam- 
ics make no time progress at this kind of equilibrium. The following definition 
distinguishes those hybrid equilibria which are created from equilibria of vector 
fields in H in the standard sense. 




434 



S. Simic et al. 



Definition 12. A point x G Mh is called a standard equilibrium for if it is 
a hybrid equilibrium and for each p G if p G Di, then p is an equilibrium 

for Xi (i.e. Xi{p) = Q). It is called a pure equilibrium if it is standard and 
belongs to Ti{intD). 

Note that the only dynamics involved in creating a pure equilibrium are those 
of a single vector field. We now define the notions of (Lyapunov) stability and 
asymptotic stability of hybrid equilibria in analogy with those from dynamical 
systems. 

Definition 13. An equilibrium is called (Lyapunov) stable if for every 

neighborhood U of a;* in Mh there exists a neighborhood V of x* in U such that 
for every x G V, G U for all t G [0, Too(a;)). If V can be chosen so that 

in addition to the properties described above, lim(^^^( 2 .) = a;*, then x» is 

asymptotically stable. 

Example 17. There are well known 2-dimensional hybrid systems (and they are 
also not difficult to construct from scratch; cf. [SJSL]) with a standard hybrid 
equilibrium which can described as follows: stable -I- stable = unstable, or un- 
stable -I- stable = stable, or unstable -I- unstable = stable. This means that (in 
the case of the first example) the unstable hybrid equilibrium in question is cre- 
ated by stable equilibria for the vector fields at play in the hybrid system. These 
examples show us that extra caution is needed in analyzing stability of hybrid 
equilibria. 

In the subsequent text, we use the following notation: if X is a vector field 
on a manifold M with local flow ft and / : M ^ R a function, Xf will denote 
the derivative of / in the direction of X: (Xf){x) = Tf{X{x)). For a map h : 
{A, dA) {B, ds) between metric spaces, let Lipp(/) = supg(,A-{p} 

This is the Lipschitz constant of f at p. 

The following theorem is an analog of the linearization theorem for stability 
of equilibria of a single dynamical system. In the hybrid case, the linearized 
data include, besides the derivatives of the vector fields at the equilibrium, the 
tangent spaces at the equilibrium of guards and images of resets involved in the 
hybrid dynamics near the equilibrium. Here, for a manifold A with boundary 
and p G dA, we denote by A the set of all vectors v G TpA which point inside 
A (i.e. there exists e > 0 and a smooth curve c : [0, e] ^ H such that c(0) = p, 
c(0) = V and c(t) G A — dA for 0 < t < e). 

Theorem 9 (Stability via Linearization). 

Let x» G Mh be an isolated standard equilibrium for and 7r“^(a;*) = 
{pi, . . . ,pi}, where pj G Di. and 1 < j <1. Suppose that there exists a bounded 
neighborhood W of x^, and for each 1 < j < ? a smooth function fj : Uj — {pj\ — > 
K, where Uj is a neighborhood of Di. n7r“^(IT) in {ij} x R”, such that: 

(a) pj G Aj n Bj, where Aj = \m.R(^i._.^j.) n Uj, Bj = G(ij,ijj-i) n Uj, for all 

1 < j <1. Assume further that Aj and Bj are differentiable at pj. 
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(b) a- < fj < Oj" on Aj, and Bj = for all j, for some numbers 

aj <a+ < bj. 

(c) 0 < to” < Xi.fj < TO+ on Uj - {pj} (l< j < 1). 

(d) For each j there exists tj > 0 such that {T^.Aj) C where Lj = 

Tp.X^r 

For 1 < j <1, let Sj be annx (n— 1 ) -matrix whose columns form an orthonormal 
basis for Tp.Aj and belong to T^.Aj. Let 




and Vj = Define rjuix^t) = ^/?7H(a;*) < 1, then x* is 

an asymptotically stable hybrid equilibrium. //dimH = 2 and pu{xt) > 1, then 
X* is unstable. 

Remarks. 

(i) Condition (b) says that Bj is the closure of a level set of fj while Aj is 
“almost” a level set of fj . The function fj measures the progress trajectories 
of Xi^ make towards Bj, starting from Aj. 

(ii) Condition (c) says that the time-Tj map of the linearization of the flow of 
Xi- at Pj (i.e. Tfff) maps T^^Aj to T^^Bj. This means that at least on the 
level of linearizations, Bj is reachable from Aj in a bounded amount of time. 

(iii) Note that (unlike in [B] and [MH]) it is not necessary to integrate any 
vector fields and that all the input data of the theorem are computable 
(even though finding fj’s and Tj’s may be difficult). 



Example 18. Define a 3-dimensional hybrid system H by: Di = {1} x S, D 2 = 
{2} X R3 _ where S' = {(x, y, z) : a; > 0, y > , z G K} U {(x, y, x) : x < 

0, y > — x(x — c), z G M}, and G(l, 2) = {(x, y, z G Z?i : y = x^}, G(2, 1) = 
{(x,y,z) G D 2 : y = — x(x — c)}, for some constant c. Let Xi(x,y, z) = (— x — 
y,x — y, — Aiz) and X 2 {x, y, z) = {x — y,x y, A2Z), where 0 < A2 < 1 < Ai. 
Then it is not difficult to check that ?7 h( 0) = where 7 = arctanc, so if 

c > 0, then 0 is asymptotically stable. 



Example 19. Let H be a 3-dimensional hybrid system with Di = {1} x K x 
K and D 2 = {2} x — K x M, where K = [0,oo) x [0,oo). Let G(l,2) = 

{(x, y,z) e Di : X = 0}, G(2, 1) = {(x, y, z) G £>2 : y = 0}, and Xi{x, y, z) = 
(x — y, X -I- y, — Aiz), ^2(x, y, z) = (— x — y,x — y, A2Z), where Ai, A2 > 0. The 
resets are identity maps. 

Then the full trajectories of Xi are spirals around the z-axis which increase 
in radius and converge to the xy-plane. The full trajectories of X2 are also spirals 
around the z-axis, but they decrease in radius and diverge from the xy-plane. 
It is not difficult to check that, with notation from Theorem 9, = e"/"^, 

fj ,2 = , so ?7 h (0) > 1 and the theorem is inconclusive. 
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However, the flows can be decoupled into their xy- and 2 -parts the analysis 
of which shows that if Ai > 3 A 2 , then 0 is an asymptotically stable hybrid 
equilibrium of H. The reason Theorem 9 does not provide the same answer, 
intuitively speaking, is because it is not able to measure the small amount of 
contraction around 0 in the flows of both Xi and X 2 , which turns out to be 
sufficient for asymptotic stability. Namely, on G(2, 1) the flow of X\ contracts 
in only one direction (and expands in the other) and similarly for the flow of X^ 
on G(l, 2). 
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Abstract. An algorithm for computing the maximal controlled invari- 
ant set and the least restrictive controller for discrete time systems is 
proposed. We show how the algorithm can be encoded using quanti- 
fier elimination, which leads to a semi-decidability result for definable 
systems. For discrete time linear systems with all sets specified by lin- 
ear inequalities, a more efficient implementation is proposed using linear 
programming and Fourier elimination. If in addition the system is in 
controllable canonical form, the input is scalar and unbounded, the dis- 
turbance is scalar and bounded and the initial set is a rectangle, then 
the problem is decidable. 



1 Introduction 

The design of controllers is one of the most active research topics in the area 
of hybrid systems. Problems that have been addressed include hierarchical con- 
trol [5, 19], distributed control [18], and optimal control using dynamic program- 
ming techniques [3, 4, 20, 23] or extensions of the maximum principle [11]. A 
substantial research effort has also been directed towards solving control prob- 
lems with reachability specifications, that is designing controllers that guarantee 
that the state of the system will remain in a “good” part of the state space. Such 
control problems turn out to be very important in applications, and are closely 
related to the computation of the reachable states of a hybrid system and to the 
concept of controlled invariance. The proposed solutions extend game theory 
methods for purely discrete [21, 25] and purely continuous [2, 15] systems to 
certain classes of hybrid systems: timed automata [13, 17], rectangular hybrid 
automata [28] and more general hybrid automata [16, 26]. 

All of these techniques are concerned with hybrid systems whose continuous 
state evolves in continuous time, according to differential equations or differential 
inclusions. Unlike conventional continuous dynamical systems, little attention 
has been devoted to systems where the continuous state evolves in discrete time, 
according to difference equations. Besides being interesting in its own right, 
this class of hybrid systems can be used to approximate hybrid systems with 
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differential equations. Indeed, most of the techniques that have been proposed 
for reachability computations for general continuous dynamics involve some form 
of discretization of the continuous space [8, 12, 26], followed by a reachability 
computation on the resulting discrete time system. 

In Sect. 2, we formulate the problem of controller synthesis for discrete time 
systems under reachability specifications, introduce the concepts of maximal 
controlled invariant set and least restrictive controller, propose an algorithm for 
computing them, and show how the algorithm can be implemented using quanti- 
fier elimination. This immediately leads to a semi-decidability result for discrete 
time systems whose continuous dynamics can be encoded in a decidable theory 
of the reals. In Sect. 3, we implement the proposed algorithm for discrete time 
linear systems with all the sets defined by linear inequalities. The implemen- 
tation is based on a more efficient method for performing quantifier elimina- 
tion in the theory of linear constraints using linear programming and Fourier 
elimination. We also show that the problem is decidable when the single-input 
single-disturbance discrete time linear system is in controllable canonical form, 
the input is unbounded, and the safe set is a rectangle. Finally, in Sect. 4, we 
illustrate the proposed method with some examples. For the proofs we refer the 
reader to [27]. 

2 Discrete Time Systems and Safety Specifications 

2.1 Basic Definitions 

Let F be a countable collection of variables and let Y denote its set of valuations, 
that is the set of all possible assignments of these variables. We refer to variables 
whose set of valuations is countable as discrete and to variables whose set of 
valuations is a subset of a Euclidean space M” as continuous. For a set Y we use 
Y'^ to denote the complement of Y, 2^ to denote the set of all subsets of Y, Y* 
to denote the set of all finite sequences of elements of Y, and Y“ to denote the 
set of all infinite sequences. Since the dynamical systems we will consider will 
be time invariant we will use y = {y[i]}^Q to denote sequences. We use A to 
denote conjunction, V to denote disjunction, ^ to denote negation, V to denote 
the universal quantifier, and 3 to denote the existential quantifier. 

Definition 1 (Discrete Time System (DTS)). A discrete time system is a 
collection H = (Y, y,Init,/) consisting of a finite collection of state variables, 
X, a finite collection of input variables, V, a set of initial states, Init C X, and 
a reset relation, / : X x V ^ 2^. 



Definition 2 (Execution of DTS). A sequence x = {x, v) € (X x V)* U (X x 

V)“ is said to be an execution of the discrete time system H if x[0] € Init, and 
for all k>0, s[fc-|- 1] € f{x[k],v[k]). 
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To ensure that every finite execution can be extended to an infinite execution we 
assume that f{x, v) 0 for all {x, v) G X x V. We call such a DTS non-blocking} 
We denote the set of all executions of H starting at a;o G X as Sh{xo), and 
the set of all executions of H by Sh- Clearly, Sh = [Jxoeinit^H{xo)- 

Our goal here is to design controllers for DTS. We assume that the input 
variables are partitioned into two classes, V = U U D, where U are control 
variables, and D are disturbance variables. In this context a controller can be 
defined as a feedback map. 

Definition 3 (Controller). A controller, C, is a map C : X* ^ 2^ . A con- 
troller is called non-blocking if C{x) yf 0 for all x G X*. A eontroller is called 
memoryless if for all x,x' G X* ending at the same state we have C(x) = C(x'). 

The interpretation is that, given the evolution of the plant state up to now, the 
controller determines the set of allowable controls for the next transition. With 
this interpretation in mind, we define the set of closed loop causal executions as 

£hc = {{x, u, d) e Sh \ yk> 0, u[k] G C{xik)}, 

where xlk denotes the subsequence of x consisting of its first k elements. Notice 
that a memoryless controller can be characterized by a map g : X ^ 2^, and 
its set of closed loop causal executions is simply 

= {(a;, u, d) e Sh \ ^k > 0, u[k] G g{x[k])}. 

Our goal is to use controllers to steer the executions of the plant, so that they 
satisfy certain desirable properties. In this paper we will restrict our attention 
to a class of properties known as safety properties: Given a set T’ C X, we would 
like to find a non-blocking controller that ensures that the state stays in F for 
ever. We will say that a controller C solves the problem {H, OF), if and only if 
C is non-blocking and for all (x,u,d) G Shc: x[k] G F for all fc > 0. If such a 
controller exists we say that the problem {FI, OF) can be solved. 

Even though safety properties are not the only properties of interest^, they 
turn out to be very useful in applications. Many important problems, such as 
absence of collisions in transportation systems, mutual exclusion in distributed 
algorithms, etc., can be naturally encoded as safety properties. Fortunately, it 
can be shown that for this class of properties one can, without loss of generality, 
restrict attention to memoryless controllers. 

Proposition 1. The problem {H, DF) can be solved if and only if it can be 
solved by a memoryless controller. 

Motivated by Proposition 1, we restrict our attention to memoryless con- 
trollers from now on. 

^ The condition is only sufficient. Althongh it can be refined to be necessary as well, we 
will not pnrsue this direction since the emphasis of this paper is controller synthesis. 
^ Other important properties are liveness properties (ensuring that the state eventually 
reaches a certain set, visits a set infinitely often, etc.), stability, optimality, etc. 
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2.2 Controlled Invariant Sets and Least Restrictive Controllers 

The concept of controlled invariance turns out to be fundamental for the design 
of controllers for safety specifications [16]. Roughly speaking, a set of states, 
W, is called controlled invariant if there exists a controller that ensures that all 
executions starting somewhere in W remain in W for ever. More formally: 

Definition 4 (Controlled invariant set). A set IT C X zs called a controlled 
invariant set of H if there exists a non-blocking controller that solves the problem 
{H' , OW), where H' = {X, V, W, f) (the same as H , but with Init' = W). 

We say that the controller that solves the problem {H' , UW) renders the set W 
invariant. Also, given a set T C X, a set IT C F is called a maximal controlled 
invariant subset of F, if it is controlled invariant and it is not a proper subset of 
any other controlled invariant subset of F. The following lemma establishes the 
uniqueness of the maximal controlled invariant set. 

Lemma 1. The problem {FI, OF) can be solved if and only if there exists a 
unique maximal controlled invariant set, W , with Init CWCF. 

A useful and intuitive characterization of the concept of controlled invariance 
can be given in terms of the operator Pre : 2^ — > 2^ defined by 

Pre(lT) = {a; G IT I e U Vd G D, f{x, u, d) n 1T° = 0} . 

The following properties of the operator Pre are easy to establish and will be 
useful in the subsequent discussion. 

Proposition 2. The operator Pre has the following properties: 

1. Pre is contracting, that is for all IT C X, Pre(lT) C W; 

2. Pre is monotone, that is for all IT, IT' C X with IT C W' , Pre(IT) C 
Pre(IT'); and, 

3. A set IT C X is controlled invariant if and only if it is a fixed point of Pre, 
that is if and only i/Pre(IT) = IT. 

Many memoryless controllers may be able to solve a particular problem. 
Controllers that impose less restrictions on the inputs they allow are in a sense 
better than controllers that impose more restrictions. For example, controllers 
that impose fewer restrictions allow more freedom if additional safety specifica- 
tions are imposed, or if one is asked to optimize the performance of the (safe) 
closed loop system with respect to other objectives. To quantify this intuitive 
notion we introduce a partial order on the space of memoryless controllers. We 
write gi ^ g 2 if for all x G X, gi(x) C g 2 {x). 

Definition 5 (Least restrictive controller). A memoryless controller g : 
X — > 2^^ that solves the problem {H, F) is called least restrictive if it is maximal 
among the controllers that solve {H, DF) in the partial order defined by F. 
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Lemma 2. A controller that renders a set W invariant exists if and only if a 
unique least restrictive controller that renders W invariant exists. 

Notice that the least restrictive controller that renders a set W invariant must, 
by definition, allow g{x) = U for all x W . Summarizing Lemmas 1 and 2 we 
have the following: 

Theorem 1. The problem {H, □.F) can he solved if and only if there exists: 

1. a unique maximal controlled invariant set W with Init CWCF, and 

2. a unique least restrictive eontroller, g, that renders W invariant. 

Motivated by Theorem 1 we state the controlled invariance problem more 
formally. 

Problem 1 (Controlled Invariance Problem (CIP)) Given a DTS and a 

set F C X eompute the maximal controlled invariant subset of F, W , the least 
restrictive controller, g, that renders W invariant, and test whether Init C W . 



2.3 Computation of W and g 

We first present a conceptual algorithm for solving the CIP for general DTS. 
Even though there is no straightforward way of implementing this algorithm in 
the general case, in subsequent sections we show how this can be done for special 
classes of DTS. 



Algorithm 1 (Controlled Invariance Algorithm) 



initialization: = F, W ^ = X, ^ = 0 

while W’'~^ n {W^Y 0 do 
p//+i ^ Pre(lT0 
l = l+l 

end while 

set W = 

set g{x) ^ I {« e U I Vd G D, f{x, u, d) D (W)^ = 0} 



xew 

x^W 



Theorem 2. W is the maximal controlled invariant subset of F and g is the 
least restrictive controller that renders W invariant. 

To implement the controlled invariance algorithm one needs to be able to 
(1) encode sets of states, perform intersection and complementation, and test 
for emptiness, (2) compute the Pre of a set, and (3) guarantee that a fixed point 
is reached after a finite number of iterations. For classes of DTS for which 1 and 
2 are satisfied we say that the CIP is semi- decidable; if all three conditions are 
satisfied we say that the CIP is decidable. As an example, consider finite state 
machines (FSM), that is the class of DTS for which X, U and D are finite. In 
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this case, one can encode sets of states, perform intersection, complementation, 
test for emptiness and compute Pre by enumeration (or other more efficient 
representations). Moreover, by the monotonicity of and the fact that X is 
finite, the algorithm is guaranteed to terminate in a finite number of steps. 
Therefore, the CIP is decidable for finite state machines. 

In subsequent sections we show how the computation can be performed for 
DTS with state and input taking values on a Euclidean space and transition 
relations given by certain classes of functions of the state and input. 

2.4 CIP for Definable Discrete Time Systems 

In this section we consider the case where all the sets involved in the CIP can be 
expressed by means of a logic formula that belongs to the language of a certain 
logic theory. For example, we denote by Lin(R) the theory of linear constraints 
and by OF(IR) the theory of polynomial constraints. 

For some theories, it is possible to determine the sentences that belong to the 
theory. The Tarski-Seidenberg decision procedure provides a way of doing this for 
OF(M). It can be shown that OF(R) is decidable [22, 24], in other words, there 
exists a computational procedure that after a finite number of steps determines 
whether an 7^-sentence belongs to OF(M) or not. The decision procedure is based 
on quantifier elimination, an algorithm that converts a formula . . . , x„) to 
an equivalent quantifier free formula. Notice that this provides a method for 
testing emptiness. A set Y = {(a;i, . . . , x„) | 4>{xi , . . . , x„)} is empty if and only 
if the sentence 3x\ . . . 3xn \ 4>{xi , . . . , x„) is equivalent to false. 

To relate this to the problem at hand, we restrict our attention to CIP which 
are “definable” in an appropriate theory. 

Definition 6 (Definable CIP). A CIP, {H,nF), is definable in a theory if 
X = M", U C , D C and the sets U, D, Init, f{x,u,d) for all a: € X, 
M S U and c? G D, and F are definable in the theory. 

If {H, UF) and IT* are definable in OF(K), then 

V>'(a;) = BuVdVx' \ [xGiT*] A [mGU] A [(d^D) V {x' ^f{x,u,d)) V (x'gIT*)] (1) 

is a first order formula in the corresponding language. Therefore, each step of 
the controlled invariance algorithm involves eliminating the quantifiers in (1) to 
obtain a quantifier free formula defining IT*+*^. The fact that OF(R) is decidable 
immediately leads to the following: 

Theorem 3. The class of CIP definable in OF(R) is semi- decidable. 

Moreover, if {H, DF) is definable in OF(M) and IT is a controlled invariant set 
also definable in OF(M), then the set {{x,u) | Vd G D \/x' G f{x,u,d), x' G 
IT} describing the least restrictive controller that renders IT invariant is also 
definable in OF(R). Furthermore, quantifier elimination can be performed in 
this formula, to obtain an explicit expression for the least restrictive controller. 
Finally, the question IT C Init° = 0 can be decided. Therefore, if the algorithm 
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happens to terminate in a finite number of steps, the CIP can be completely 
solved. 

Although different methods have been proposed for performing quantifier 
elimination in OF(R) [1, 22, 24], and the process can be automated using sym- 
bolic tools [9], the quantifier elimination procedure is in general hard, both in 
theory and in practice, since the solvability may be doubly exponential [14]. For 
the theory Lin(M), a somewhat more efficient implementation can be derived 
using techniques from linear algebra and linear programming. The next section 
shows how quantifier elimination in the theory Lin(M) can be performed more 
efficiently for the formula ( 1 ) used in the controlled invariance algorithm. 

3 CIP for Discrete Time Linear Systems 

A linear CIP (LCIP) consists of 

- a Linear DTS (LDTS), i.e. a DTS with X = K”, U = {u G \ Eu < tj} C 

D = {d G 1 Gd < 7 } C Init = {a; G X j Ja; < 0} and a reset 
relation given by /(a;, u, d) = {Ax+Bu+Cd}, where A G Q”^", B G 
C G E G G G 77 G Q™“, 7 G J G 

and 9 G with m„, mj, and rrii being the number of constraints on the 
control, disturbance and initial conditions, respectively; and, 

— a set F = {x G M" j Mx < /?} where M G fj £ Q™ and m is the 

number of constraints on the state. 

Notice that LDTS are non-blocking and deterministic, in the sense that for every 
state x and every input (u, d) there exists a unique next state. Since the sets 

U and D are all convex polygons, and the dynamics / are given by a linear 
map, the LCIP is definable in the theory Lin(K), and therefore, according to the 
discussion in Sect. 2.4, it is semi-decidable. We assume that the sets F and U 
can be either bounded or unbounded, but D is bounded^. 

For the LCIP it turns out that, after the Lth iteration, the set can be 
described by m} linear constraints as {x G M" ] M^x < /?*}, that is, IF* remains a 
convex polygon. Obviously, mP = m, = M and /3° = /3. Letting A* = M^A, 
B^ = M^B and (7* = M^G, (1) becomes 

= [M^x < /?*] A[3u 1 {Eu < T])A{\/d \ {Gd > -f)V{A^x + B^u + &d < /?*))]. 



Thus, in each step of the algorithm, we need to be able to eliminate variables 
u and d from the inner formulae, intersect the new constraints with the old ones 
and check if the new set is empty. Notice that not all of the new constraints 
generated by quantifier elimination may be necessary to define the set IF*“'"*^. 
Also, some of the old constraints may become redundant after adding the new 
ones. Hence we need to check the redundancy of the constraints when doing the 
intersection. 

® The theoretical discussion can be extended to unbounded D sets, but the computa- 
tional implementation is somewhat more involved. 
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3.1 Quantifier Elimination 

We first perform quantifier elimination on d over the formula 

(/)' {x,u)=\/d \{Gd> j)y {A^x + I3^u + C^d< . 

Let df, of and cf be the z-th row of Af 13^ and Cf respectively. Then, parsing 
6’' leads to 

4>\x,u) = Vd I f\{Gd > 7) V {cfd < p\ — dfx — bfu). 

i=l 

Consider d : ^ R'"' defined by SACP = max (cf d) for i = 1, . . . , m'. 

^ ' d-.Gd<j * ^ ’ ’ 

Proposition 3. (p{x,u) is equivalent to q}’‘{x,u) = A^x + &u < /3* — 5{&). 

Therefore, the elimination of the V quantifier can be done by solving a finite 
collection of linear programming problems. Since we have assumed that D is 
bounded, such an optimization problem is guaranteed to have a solution, and 
hence d(-) is well defined. Since d(-) is applied to each row of &, in the sequel 
we will use 5i{&) and 5{cf) interchangeably. Notice that, strictly speaking, d(-) 
is not part Lin(M), but we use it as a shorthand for the constant obtained by 
solving the linear programs. 

Next, we perform quantifier elimination on u over the formula 

p\x) = 3u I {Eu <t])A (i'x + &u < /?' - d(C')). (2) 

We will discuss two methods to eliminate u. The first is known as Fourier Elimi- 
nation [10], and the second, attributed to Cernikov [6], is an application of Farkas 
Lemma on duality [7]. 

For the first method, assume we want to eliminate u\ first. Let be the z-th 
unit vector in 

and 

Thus (A{x) is equivalent to 3u \ Fd^u < ^\x). Also define = {p\ Flpi > O}, 
Q* = {g I FIgi < 0} and i?* = {r | Fd^ = O}, where ddf refers to the i,j element 
of the matrix ddK Then (j>Ax) is equivalent to 

^ m ^ m 

I A A TTT (??(^) - H ^ ^ TTT (4(^) " H 

pdPlqdQl [ 9I 3=2 pi 3=2 

m 

A A 0<(^'(a;)-^id^zz,) 

refl' i=2 
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Hence, after the elimination of ui we obtain 





- (hU -Ki) 






(3) 



Therefore, the elimination of the 3 quantifier is performed by taking nonnegative 
linear combinations of all pairs of constraints so as to cancel the quantified 
variable. Note that if all the coefficients of the quantified variable are positive 
(negative), then <j/ is true, and we need not to eliminate the remaining variables. 
Otherwise, after u\ has been eliminated, we apply the same procedure to the 
constraints in (3), so as to eliminate U 2 , ■ ■ ■ ,Un^. Since the procedure is based 
on nonnegative row operations, it is clear that 



cj)\x) = A^ 



f A^x 

V 0 






(M'x < /?') A (0 < A^ri) , (4) 



where A^ = [A\ Hy G Qm'x(in‘+mu) jg matrix with nonnegative entries such 
that A^H^ = 0, fh^ is the number of new constraints obtained through quantifier 
elimination, = A[A'‘ G Q-m‘xn pi _ g Q'"’. Notice that if 

the condition > 0 is violated, then W = %. Otherwise, we just need to add 
the new constraints M^x < to the original set . 

Although Fourier Elimination is attractive because of its simplicity, it is quite 
inefficient. In general, it generates many new constraints in the intermediate 
steps, and in the worst case the method is exponential. This difficulty can be 
partially remedied since many of the inequalities are likely to be redundant [7] . 

An alternative method [6] computes the rows of A^ directly as the extreme 
points of the set {A' G | = 0 A A' > 0 A X) A' = 1}, where 

the last constraint is added to ensure that the set is a polytope. Although the 
extreme points method is better than Fourier elimination, because it eliminates 
the costly intermediate steps, the computation of the extreme points is still costly 
and also generates a lot of redundant constraints. A more efficient method [14] 
uses a generalized linear programming formulation and an on-line convex hull 
construction to obtain an incremental inner approximation of the set defined 
by (fi . The method considerably reduces the number of constraints defining the 
resulting set. 



3.2 Intersection, Emptiness and Redundancy 

Provided that > 0, the quantifier elimination procedure presented above 
computes the set of states IF* = {x \ M’-x < /?*} that can be forced by u to tran- 
sition into IF*. To obtain IF*“*'^, such a set must be intersected with IF*. Since 
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both sets are convex, the intersection can be carried out by simply appending 
M* and to and respectively. However, this method of performing the 
intersection is likely to lead to a description of the set which is larger than nec- 
essary since many of the constraints may be redundant. Algorithm 2 is aimed 
at checking the emptiness of the intersection and then eliminate redundant con- 
straints. In the algorithm, [] denotes an empty matrix, 1 = (1 . . . 1)^ G Q™ , 



and m'^ and /?' are the t-th rows of Mg = 



M' 

M' 



and /Sg = 



, respectively. 



Initially, M' = Mg and /?' = /3g. 

The idea behind the algorithm is that yf 0 if and only \i3x\M'x < j3' , 

which is equivalent to saying that min{t | M'x < (3' + It} < 0. Afterwards, if 
the problem maxjm'^a; | M'x < /?'} is feasible, and the constraint m'^ x < 
is not redundant, then the optimal value of the problem is /?'. Moreover, if the 
non-redundant constraint m'^ x < /?' is removed from the optimization problem, 
then the new optimal value m* satisfies m* > /?'. 



Algorithm 2 (Emptiness and Redundancy Algorithm) 

initialization M' = Mg, /?' = /3g, M*+^ = [], = []. 

m* = minjt | M'x < P' + It} 

if m* > 0 or ^ 0 then 

W = %, terminate controlled invariance algorithm 

else 

for z = 1 to -I- do 

remove m'^ from M' and /3( from P' 
m* = maxjm'^a: | M'x < P'} 
if m* > /?' then 

add m'^ to M*+^ and M', 
add /?' to and P' 
end if 
end for 
end if 

if M^+^ = M^ and = /3* then 

W = W\ terminate controlled invariance algorithm 

end if 

The controlled invariance algorithm terminates if the redundancy algorithm 
concludes that either A^ry ^ 0 or = 0 (in which case W = 0), or if all the 

new constraints are redundant (in which case = W^. Otherwise, 

upon termination of the redundancy algorithm, the process is repeated for . 
An obvious optimization of the code involves terminating both algorithms if after 
all new constraints in M^x < have been tested, M*+^ and are still empty. 
Notice that for all I the set IT* is a convex polygon as claimed. Summarizing: 

^ Note that any redundant constraint in the original description of F will be elimi- 
nated the first time the redundancy algorithm is invoked by the controlled invariance 
algorithm. 
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Theorem 4. The LCIP is semi-decidable. 

In the next section we study situations where the algorithm is guaranteed to 
terminate in a finite number of steps. In Sect. 4, we will provide and example 
which actually converges after an infinite number of iterations. 



3.3 Decidable Special Cases 

We first summarize some of the observations made so far about situations where 
the algorithm terminates in a finite number of steps. 

Proposition 4. For an LCIP with U = if either one of the columns of 
MB is componentwise positive (negative), or if rank{MB) = min{m,n}, the 
algorithm terminates in a finite number of steps. 

Next, we limit our attention to the case F = [ai, (3i] x ... x [an, fin] C M" 
with Oj < Pi and [ai,Pi] C M., i = 1 . . . n, u G R, and d G [di, ^ 2 ] C R. To remind 
ourselves of the fact that u and d are scalar, we use b and c instead of B and C. 
We also assume that {A, b) is in controllable canonical form, that is 



x[k -1- 1] = 


/ 0 10 0- 

0 0 10- 


•• 0 \ 
• • 0 


x[k] + 


... 00 


u[k] + 


( ^ 

C2 


d[k]. (5) 




0 

\ Uni Un2 * * ‘ 


1 

^nn j 




0 

v) 




Cn— 1 

V C„ / 





In this case tp^{x) is equivalent to 

n n 

3u I f\ (aj < Xj < Pj) A f\ (aj-i - < Xj < Pj-i - <5(cj_i)) A 

i=i i=2 

( n n \ 

an-'Y^ anjXj - 6{-Cn) < u < Pn ~ Y 

i=i i=i J 



From the last expression, it is clear that given xi G [ai,Pi], Xj exists if and only 
if a] = ma,x{aj,aj-i — 6{—Cj-i)) < min(/3j, /3j_i — (5(cj_i)) = /3j, j = 2 .. .n, 
and u exists if and only if a„ — i5(— c„) < /?„ — <5(c„). It is straightforward to 
see that in the Ath iteration {0 < I < n) is defined by: 



W — [a /3]^] X ... X /3;_i_]^] x [o;+ 2 jA+ 2 ] x ... x [«„,/?„], 



where a' = max(a'“^, - i5(cj_i)), and /3j = mm{p''~^ , PjZ\ ~ <^(cj-i)), for 

2<l-\-l<j<n, with aj = aj and /3° = Pj, for 1 < j < n. 

This means that after n iterations, the maximal controlled invariant set re- 
mains unchanged, and the least restrictive controller is given by the last con- 
straint in (6), but with a„ and /?„ replaced by respectively. This 

result can be summarized as follows: 
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Lemma 3. Given system (5) with F = [ai, /3i] x . . . x [a„, /3„] C R", U = R and 
D = [di,d2] C R, the solution to the CIP, obtained after at most n iterations of 
the algorithm, is given by: 



W = 



g{x) = 



'{xIA" 



= 1 



-^} ^A;=2(«r </5r)A( 

otherwise 



|Cn| < j 



I a” ^ - 5{-Cn) <u + YTj=\(^rijXj < jd'f ^-<5(c„)| ifxew 
U otherwise 



Theorem 5. For systems of the form (5) with F = [a\, j3\]x . . .x [a„, /?„] C R", 
U = R and D = [tii, ^ 2 ] C R, the LCIP is decidable. 

The conditions of Theorem 5 for decidability are somewhat demanding. If, for 
example, u is bounded, that is, U = [mi, M2] C R, then the new constraints added 
to X during each iteration may change the bounds on a; to a non-rectangular 
polyhedron. In this case, the CIP is no longer decidable, and the system falls 
into the more general class of systems described at the beginning of the section. 
We conjecture that the LCIP is decidable in a much more general setting, using a 
completely different algorithm that exploits the stabilizability of the pairs {A, B) 
and {A, C) and the observability of the pair {A, M). 



4 Experimental Results 



The algorithm proposed in Sect. 3 was implemented in MATLAB. Here, we 
present two examples that were solved using this implementation. The first ex- 
ample is also worked out analytically to complete the semi-decidability result. 



Example 1. The LDTS is defined by U = R, D = [— 1, 1], 



A = 





,M = 



-1 -3 
1 -1 

V-3 1 ] 



, and P = 



-50 

100 

V-50/ 



It is straightforward to see that the only new constraint added in the Lth iter- 
ation is [0 mi]x < Pi, where mi = —10 • 3^“^, and Pi = —210 — 265(3*“^ — 1). 
Therefore after an infinite number of iterations, W and g(x) converge to 




r {m e U I M > max(18 — xi — ^, — 100 — xi,—^ — xi — X 2 ) 
g(x) = < M < min(98 — Xi — 2x2, —52 — xi + 2 x 2 )} 



U 



if a: G IT 
else 
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Example 2. The LDTS is defined by 



A = 




.B = 




,C = 




,E = 



0 1 

Vo-iy 



,v = 



/ 1000 \ 

1000 

1000 

\ioooy 



3 1 



-1 


3 


0 




100 




-1 


0 


0 




1 


1 


-1 


0 


,/? = 


100 


,G = 


0 


1 


0 


and 7 = 


1 


-1 


-1 


0 


100 


0 


-1 


0 


1 


-1 


0 


1 




100 




0 


0 


1 




1 


V 0 


0 


-V 








^ 0 


0 


-y 




vJ 



1 1 



M = 



Using MATLAB, this example converges in two iterations. Information about 
the intermediate calculations of each iteration is shown in Table 1. 



Table 1. Results of Example 2 



Iteration 1 2 

Number of LP problems for quantifier elimination on d 6 10 

Number of constraints on (x, u) before elimination of u 10 14 

Number of new constraints on x after elimination of u 281 614 
Number of new non-redundant constraints on x 4 0 

Total number of constraints on x after iteration 10 10 



5 Conclusions and Future Work 

We showed that the problem of computing the maximal controlled invariant 
set and the least restrictive controller for discrete time systems is well posed 
and proposed a general algorithm for carrying out the computation. We then 
specialized the algorithm to discrete time linear systems with convex polygonal 
constraints, and showed how it can be implemented using linear programming 
and Fourier elimination. The decidability of the problem was also analyzed, and 
some simple, but interesting cases were found to be decidable. 

We are currently working on sufficient conditions under which the problem is 
decidable. So far, it seems that the decidability property is not only dependent 
on the system itself, but also on the initial set, as shown by Example 1. Another 
topic of further research, is the application of these algorithm to discrete time 
hybrid systems, where some states and inputs take values in finite sets, while 
others in subsets of a Euclidean space. It is easy to show how this class of systems 
is a special case of the more general class of DTS. Therefore, all the conclusions 
of Sect. 2 directly extend to them. Unfortunately the implementation of the 
controlled invariance algorithm is more complicated, even in the case where the 
continuous state evolves according to a linear difference equation. 
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Abstract. Results from classical dynamical systems are generalized to 
hybrid dynamical systems. The concept of w limit set is introduced for 
hybrid systems and is used to prove new results on invariant sets and 
stability, where Zeno and non-Zeno hybrid systems can be treated within 
the same framework. As an example, LaSalle’s Invariance Principle is 
extended to hybrid systems. Zeno hybrid systems are discussed in detail. 
The u! limit set of a Zeno execution is characterized for classes of hybrid 
systems. 



1 Introduction 

Systems with interacting continuous-time and discrete-time dynamics are used 
as models in a large variety of applications. The rich structure of such hybrid 
systems allow them to accurately predict the behavior of quite complex sys- 
tems. However, the continuous-discrete nature of the system calls for new sys- 
tem theoretical tools for modeling, analysis, and design. Intensive recent activity 
have provided a few such tools, for instance, Lyapunov stability results [1,14]. 
However, as will be shown in this paper, in many cases the results come with 
assumptions that are not only hard to check but also unnecessary. There are 
several fundamental properties of hybrid systems that have not been sufficiently 
studied in the literature. These include questions on existence and uniqueness 
of executions, which have only recently been addressed [12,7]. Another question 
is when a hybrid system exhibits an infinite number of discrete transitions dur- 
ing a finite time interval, which is referred to as Zeno. The significance of these 
questions has been pointed out by many researchers, e.g.. He and Lemmon [3] 
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write “An important issue [ . . . ] concerns necessary and sufficient conditions for 
a switched system to be live, deadlock free, or nonZeno.” 

The main contribution of the paper is to carefully generalize concepts from 
classical dynamical systems like oj limit sets and invariant sets, in a way so that 
Zeno executions are treated within the same framework as regular non-Zeno 
executions. It is then straightforward to extend existing results, for instance, 
Lyapunov stability theorems for hybrid systems [1,14]. We illustrate this by 
proving LaSalle’s Invariance Principle for hybrid systems. 

Zeno is an interesting mathematical property of some hybrid systems, which 
does not occur in smooth dynamical systems. Real physical systems are not 
Zeno. Models of physical systems may, however, be Zeno due to a too high level 
of abstraction. In the latter part of the paper, we characterize Zeno executions 
and their Zeno states, where the Zeno states are defined as the w limit points of 
a Zeno execution. We are able to completely characterize the set of Zeno states 
for a few classes of hybrid systems. It is shown that the features of the reset 
maps are important. For example, if the resets are identity maps or the resets 
are contractions, the continuous part of the Zeno state is a singleton. 

The outline of the paper is as follows. In Section 2 notation and some basic 
definitions of hybrid automata and executions are introduced. Some recent re- 
sults on existence and uniqueness of executions for classes of hybrid automata 
are also given. Section 3 introduces invariants sets and uj limit sets for hybrid 
automata and gives a generalization of LaSalle’s Invariance Principle. Finally, 
results on Zeno hybrid automata are given in Section 4, where for instance the 
UJ limit set for Zeno executions are discussed and some necessary and sufficient 
conditions for Zenoness are given. 

2 Hybrid Automata and Executions 

2.1 Notation 

For a finite collection V of variables, let V denote the set of valuations of these 
variables. We use lower case letters to denote both a variable and its valuation. 
We refer to variables whose set of valuations is finite or countable as discrete and 
to variables whose set of valuations is a subset of a Euclidean space as continuous. 
For a set of continuous variables X with X = M” for n > 0, we assume that 
X is given the Euclidean metric topology, and use || • || to denote the Euclidean 
norm. For a set of discrete variables Q, we assume that Q is given the discrete 
topology (every subset is an open set), generated by the metric dD{q,q') = 0 
q = q' and dD{q,q') = 1 if g yf <?'. We denote the valuations of the union 
Q U X by Q X X, which is given the product topology, generated by the metric 
d((g, x), {q' , x')) = dniq, g') + Using the metric d, we define the distance 
between two sets C/i, C /2 C Q x X by d(C/i, C/ 2 ) = inf(q;,a:i)G(7i d{{qi,xi), ( 52 , 2 ^ 2 ))- 
We assume that a subset C/ of a topological space is given the induced topology, 
and we use U to denote its closure, U° its interior, dU its boundary, C/° its 
complement, \U\ its cardinality, and P{U) the set of all subsets of U. 
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2.2 Basic Definitions 

The following definitions are based on [8,4,7]. 

Definition 1 (Hybrid Automaton). A hybrid automaton H is a collection 
H = {Q, X, Init, /, Dom, Reset), where 

— Q is a finite collection of discrete variables; 

— X is a finite collection of continuous variables with X = K”; 

— Init C Q X X is a set of initial states; 

— /:Q X X ^ TX is a vector field; 

— Dom C Q X X is the domain of H;^ 

Reset : Q x X^ — ^ X X) is a reset relation. 

We refer to {q^x) G Q x X as the state of H. Unless otherwise stated, we 
introduce the following assumption, to prevent some obvious pathological cases. 



Assumption 1 |Q| < oo and f is Lipschitz continuous in its second argument. 

Note that, under the discrete topology on Q, / is trivially continuous in its first 
argument. A hybrid automaton can be represented by a directed graph (Q,if), 
with vertices Q and edges 

if = {(g, g') G Q X Q : 3a;, x' G X, (g', a;') G Reset(g, a;)}. 

With each vertex g G Q, we associate a set of continuous initial states 
Init(g) = {a; G X : (q,x) G Init}, 
a vector field /(g, •), and a set 

i(g) = {a; G X : (q,x) G Dom}. 

With each edge e = (g, g') G E, we associate a guard 

G(e) = {a; G X : 3a;' G X, (g', a;') G Reset(g, a;)}, 
and a reset map 

R(e,x) = {x' gX : {q' ,x') G Reset(g, a;)}. 

Since there is a unique graphical representation for each hybrid automaton, we 
will use the corresponding graphs as formal definitions for hybrid automata in 
most examples. 



^ The set Dom is often called the invariant set in the hybrid system literature in 
computer science. We reserve this term for later in the paper, where we will discuss 
sets invariant in the usual dynamical systems sense. 
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Definition 2 (Hybrid Time Trajectory). A hybrid time trajectory t is a 
finite or infinite sequence of intervals r = {/i}fc=o» such that 

- h = [Ti,r'] for i < N , and, if N < oo, In = [tn,t'n] or In = [tn,t'j^); and 

- Ti <tI = Ti+i for i > 0. 

A hybrid time trajectory is a sequence of intervals of the real line, whose end 
points overlap. The interpretation is that the end points of the intervals are 
the times at which discrete transitions take place. Note that Ti = r' is allowed, 
therefore multiple discrete transitions may take place at the same “time” . Since 
the dynamical systems we will be concerned with are time invariant we will 
sometimes, without loss of generality, assume tq = 0. Hybrid time trajectories 
can extend to infinity if r is an infinite sequence or if it is a finite sequence ending 
with an interval of the form [tn, oo). We denote by T the set of all hybrid time 
trajectories and use t € r as shorthand notation for that there exists i such that 
t G li G T. For a topological space K we use k : t ^ K as a, short hand notation 
for a map assigning a value from K to each t € t; note that k is not a function 
on the real line, as it assigns multiple values to the same t G M: t = t' = r^+i 
for all i > 0. Each r G T is fully ordered by the relation ^ defined by ti -< t2 
for ti G [ti, t'] and t2 G [tj, rj] if and only if z < j, or z = j and ti < t2- 

Definition 3 (Execution). An execution x of a hybrid automaton H is a col- 
lection X = {t, q, x) with t G T, g : r — > Q, and a: : t — > X, satisfying 

- (9(^-0) , a;(ro)) G Init (initial condition); 

- for all i with Ti < r', g(-) is constant and x{-) is a solution^ to the differential 
equation dx/dt = f{q,x) over [Ti,r'], and for all t G [ri,r'), {q(t),x{t)) G 
Dom (continuous evolution); and 

- for all i, (q{Ti+i) , x{Ti+i)) G Reset (g(r'), a;(r')) (discrete evolution). 

We say a hybrid automaton accepts an execution x or not. For an execution 
X = (t, q, x), we use (qo, xq) = (9(70), a^(7o)) to denote the initial state of y. The 
execution time 7”oQ(y) is defined as Too (x) = EZo('^i - "^here iV + 1 is the 
number of intervals in the hybrid time trajectory. The argument y will sometimes 
be left out. An execution is finite if t is a finite sequence ending with a compact 
interval, it is called infinite if r is either an infinite sequence or if Too(y) = 00, 
and it is called Zeno if it is infinite but Too(y) < 00. The execution time of a Zeno 
execution is also called the Zeno time. We use SH{qo,xo) to denote the set of all 
executions of H with initial condition (qo,xo) G Init, £'^{qo,xo) to denote the 
set of all infinite executions of H with initial condition (go 7 a^o) G Init- We define 
= [}{qo,xo)&nit^H{qa,xo) and = U(go.a;o)einit (9o, a^o)- To simplify the 
notation, we will drop the subscript H whenever the automaton is clear from 
the context. 

^ “Solution” is interpreted in the sense of Caratheodory. 
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2.3 Classes of Automata 

The notation previously introduced gives a convenient way to express existence 
and uniqueness of executions. 

Definition 4 (Non-Blocking Automaton). A hybrid automaton H is non- 
blocking if £'^{qo,xo) is non-empty for all (qo,xo) € Init. 

Definition 5 (Deterministic Automaton). A hybrid automaton H is deter- 
ministic if £'^{qo,xo) contains at most one element for all (qo,xo) G Init. 

Note that if a hybrid automaton is both non-blocking and deterministic, then 
it accepts a unique infinite execution for each initial condition. In [7] conditions 
were established that determine whether an automaton is non-blocking and de- 
terministic. The conditions require one to argue about the set of states reachable 
by a hybrid automaton, and the set of states from which continuous evolution 
is impossible. A state (g, x) G Q x X is called reachable by H, if there exists a 
finite execution y = {r,q,x) with r = {[ri,r']})^Q and (g(r)y), x(r)^)) = (g, x). 
We use Reach// to denote the set of states reachable by a hybrid automaton, 
and Reach// (g) the projection of Reach// to discrete state g. We will drop the 
subscript H whenever the automaton is clear from the context. The set Reach 
is in general difficult to compute. Fortunately, the conditions of the subsequent 
results will not require us to do so: any outer approximation of the reachable 
set will be sufficient. In [2,7] methods for computing such outer approximations 
using simple induction arguments are outlined. 

The set of states from which continuous evolution is impossible is given by 

Out// = {(g°,x°) G Q X X : Ve > 0, G [0,e), (g°,x(t)) ^ Dom}, 

where x(-) is the solution to dx/dt = f{q°,x) with x(0) = x°. Note that if Dom 
is an open set, then Out is simply Dom°. If Dom is closed, then Out may also 
contain parts of the boundary of Dom. In [7] methods for computing Out were 
proposed, under appropriate smoothness assumptions on / and the boundary 
of Dom. As before, we will use Out//(g) to denote the projection of Out to 
discrete state g, and drop the subscript H whenever the automaton is clear from 
the context. With these two pieces of notation one can show the following two 
results [7]. 

Proposition 1. A (deterministic) hybrid automaton is non-blocking if (and 
only if) for all (g, x) G Out n Reach, Reset(g, x) yf 0. 

Proposition 2. A hybrid automaton is deterministic if and only if for all 
(g, x) G Reach, |Reset(g,x)| < 1 and, z/Reset(g, x) yf 0, (g,x) G Out. 

We characterize the hybrid automata such that the state remains in the closure 
of the invariant along all executions. 

Definition 6 (Domain Preserving). A hybrid automaton is domain preserv- 
ing z/ Reach C Dom. 
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The following result is now straightforward. 

Proposition 3. A hybrid automaton is domain preserving if and only z/Init C 
Dom and for all {q, x) € Dom n Reach, Heset{q, x) C Dom. 

Note that the use of Reach is again not limiting. Note also that the conditions of 
the lemma do not depend on the vector field /. This is because, by the definition 
of an execution, the state can never end up outside the closure of the domain 
along continuous evolution. 

Definition 7 (Transverse Domain). A hybrid automaton H is said to have 
transverse domain if there exists a function ct : Q x X — > K continuously differ- 
entiable in its second argument, such that 

Dom = {(< 7 , a;) G Q X X : (j{q, a;) >0} 

and for all {q,x) with a{q,x) = 0, Lfa{q,x) 0. 

Here T/cr: Q xX^M denotes the Lie derivative of cr along / defined as 

Lfa{q,x) = • f{q,x) 

In other words, an automaton has transverse domain if the set Dom is closed, 
its boundary is differentiable, and the vector field / is pointing either inside or 
outside of Dom along the boundary.^ If H has transverse domain the set Outn 
admits a fairly simple characterization. 

Proposition 4. If H has transverse domain, then 

OutH = {(g, a;) G Q X X : a{q,x) <0} 

U {{q,x) G Q X X : a{q,x) = 0 and Lfa{q,x) < 0}. 

3 Invariant Sets and Stability 

We first recall some standard concepts from dynamical system theory, and dis- 
cuss how they generalize to hybrid automata. 

Definition 8 (Invariant Set). A set M C Init is called invariant if for all 
(qo,Xo) G M, {T,q,x) G SH{qo^Xo), and t Gt, it holds that (q{t),x{t)) G M. 

The class of invariant sets is closed under arbitrary unions and intersections. 
Invariant sets are such that all executions starting in the set remain in the set for 
ever. We are interested in studying the stability of invariant sets, i.e., determine 
whether all trajectories that start close to an invariant set remain close to it. 

® Under appropriate smoothness assumptions on a and / the definition of transverse 
domain can be relaxed somewhat by allowing Lja{q,x) = 0 on the boundary of 
Dom and taking higher-order Lie derivatives, until one that is non-zero is found. 
Even though many of the results presented here extend to this relaxed definition, 
the proofs are slightly more technical. We will therefore limit ourselves to the notion 
of transverse domain given in Definition 7. 
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Definition 9 (Stable Invariant Set). An invariant set M C Init is called 
stable if for all e > 0 there exists i5 > 0 such that for all (qo,xo) G Init, with 
d{{qo, xq), M) < 6, all ( t , q, x) G Sniqo, xq), and all t G t, d{{q{t) , x{t)) , M) < e. 

An invariant set is called (locally) asymptotically stable if it is stable and in 
addition there exists Z\ > 0 such that for all (qo,xo) G Init, with d{{qo,xo), M) < 
A, and all (T,q,x) G £^{qo,xo), d{{q{t) , x{t)) , M) = 0. 

Note that since t is fully ordered the above limit is well defined. The asymptotic 
behavior of an infinite execution is captured in terms of its ui limit set. 

Definition 10 (ut limit set). The uj limit point (q,x) G QxX. of an execution 
X = (t, q, x) G is a point for which there exists a sequence {6*„}))Tq, 6*„ G r, 
such that as n ^ oo, — > Too and (g(0„), x(0„)) ^ (q,x). The to limit set 
C Q X X zs the set of all u limit points of an execution y. 

The following lemma establishes a relation between ui limit sets and invariant 
sets. For convenience the assumptions on the reset relation and the domain are 
given in the graphical notation introduced in Section 2.2. 

Lemma 1. Consider a deterministic hybrid automaton H with transverse do- 
main. Assume it is domain preserving and that f{q,-) is for all q G Q. 
Furthermore, assume that for all e = {q, q') G E, R{e, •) is continuous, and 
G(e) n I{q) is an open subset of dl{q). Then, for any execution y = (T,q,x) G 
, ifx{-) is bounded, then is (i) nonempty, (ii) compact, and (Hi) invariant. 
Further, (iv) for all e > 0 there exists T G t such that d{{q{t),x{t)), S^) < e, 
t G T, for all t >T. 

Proof. See [15]. The proofs of (i), (ii), and (iv) are similar to the corresponding 
result for continuous dynamical systems [10,13]. 

The conditions of the lemma are sufficient. They can also be shown to be tight: 
one can construct hybrid automata that violate any one of the conditions of the 
lemma that accept infinite executions whose u limit set is not invariant. The 
conditions of the lemma are also sufficient to establish continuity of executions 
with respect to initial conditions, see [15]. 

LaSalle’s Invariance Principle is a useful tool when studying the stability of 
conventional, continuous dynamical systems. Lemma 1 allows us to extend this 
tool to hybrid systems. 

Theorem 1 (LaSalle’s Invariance Principle). Consider a hybrid automa- 
ton H that satisfies the conditions of Lemma 1. Assume there exists a compact 
invariant set 17 C QxX and let = I7n0ut° and 172 = I7n0ut. Furthermore, 
assume there exists a continuous function V : 17 ^ M, such that 

— for all {q,x) G fi\, V is continuously differentiable with respect to x and 
LfV {q, x) < 0; and 

— for all (q,x) G O 2 , V (Reset(( 7 , a;)) < V{q,x). 
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Define 

51 = {{q,x) G Di : LfV{q,x) = 0} 

5 2 = {{q,x) e D 2 ■■ V (Reset(g,a;)) = V{q,x)}, 

and let M he the largest invariant subset of Si U S 2 - Then, for all (qo,xo) G D 
every execution (T,q,x) G S^{qo,xo) approaches M as t ^ Too- 

Proof. Consider an arbitrary state {qo, xq) G f2 and let x= (r, q, x) G S'^{qo, xq). 
Since 17 is invariant, (q{t),x{t)) G 17 for all t G t. Since 17 is compact and V 
is continuous, V{q{t),x{t)) is bounded from below. Moreover, V{q{t),x{t)) is a 
non-increasing function of t G r (recall that r is fully ordered), so therefore the 
limit c = lim(^,-^(^) V{q(t),x(t)) exists. 

Since 17 is bounded, x is bounded, and therefore the uj limit set S^ is 
nonempty. Moreover, since 17 is closed, S^ C fl. By definition, for any (g, x) G 
S^, there exists a sequence {0n}^^Q, G r, such that ^ Too and {q{0n), x(&n)) 
(q, a;) as n — > 00. Then, 

V(q,x) = V( lim (q(0„), x(0„)) = lim V(q(0„), x(0n)) = c, 

n — »-oo n—*oo 

by continuity of V . Since S^- is invariant (Lemma 1), it follows that LfV{q,x) = 0 
if {q,x) ^ Out, and M(Reset(g, a:)) = V{q,x) if {q,x) G Out. Therefore, S^ C 
Si U S' 2 , which implies that S^ G M since S^ is invariant. Moreover, by (iv) in 
Lemma 1, the execution y approaches S^, and hence M, as t ^ Too- 

4 Zeno Hybrid Automata 

Zeno hybrid automata accept executions with infinitely many discrete transitions 
within a finite time interval. Such systems are hard to analysis and simulate in 
a way that gives constructive information about the behavior of the real system. 
It is therefore important to be able to determine if a model is Zeno and in 
applicable cases remove Zenoness. These problems have been discussed in [4,5]. 
In this section, some further characterization of Zeno executions are made. Recall 
that an infinite execution y is Zeno if Too(y) = ~ p) bounded. 

Definition 11 (Zeno Hybrid Antomaton). A hybrid automaton H is Zeno 
if there exists (qo,xo) G Init such that all executions in Eff{qo,xo) are Zeno.^ 

Example 1. The hybrid automaton in Figure 1 is Zeno. This is easily checked by 
explicitly deriving the time intervals r( — r^, which in this case gives a converging 
geometric series. Figure 2 shows an execution accepted by the automaton. 

We make the following two straightforward observations. 

^ An alternative definition is to say that a hybrid automaton is Zeno if there is at 
least one Zeno execution in £ff{qo,xo). In that case, a non-deterministic Zeno hy- 
brid automaton may accept both Zeno and non-Zeno executions, which may be an 
undesirable feature for instance in Reach set calculations. For deterministic hybrid 
automata the two definitions coincide. 
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-0.1 < xi < 0 xi := 5 4.9 < xi < 5 

*2 < 0 X2 '■= —CX2 X2 < 0 




xi ~ 0 2.9 < xi < 3 xi ■— 3 

X2 := —CX2 X2 <0 X2 := —CX2 



Fig. 1. An example of a Zeno hybrid automaton. 




Fig. 2. An example of an execution for the hybrid automaton in Example 1. 
The continuous part of the state is shown: xi (solid) and X 2 (dotted). 

Proposition 5. A hybrid automaton is Zeno only if the graph (Q,E) has a 
cycle. 

Proposition 6. If there exists a finite collection of states such that 

- (qi,xi) = (qN,XN); 

- (qi,Xi) G Reach// for some i = 1, . . . , N; and 

- {q^+l,x^+l) = Reset (gr^.a;*) for all i = I, N - 1; 

then there exists a Zeno execution. 

Zenoness is critically dependent on the reset relation. For example, if in Exam- 
ple 1 the reset maps X 2 '.= —CX 2 are replaced by X 2 ■= X 2 l{dx 2 — 1), where 
d = l/i/20a;i(To), then the time intervals t' — t/ decrease as {l/z}“g. This is a 
diverging series, so the new hybrid automaton is not Zeno. 

If the continuous part of the Zeno execution is bounded, then it has an oj 
limit point. We introduce the term Zeno state for such a point. 







*1 < 0 
*2 < 0 
xz > 0.5 
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a;i < 0 
< 0 

xz < 0.5 



xi a;i < 5 

X2 ■■= -CX2 X2 <0 

xz ■— 4 x 3(1 - xz) xz < 0.5 




X 2 := — CX 2 xi 0 xi < 5 X 2 '■= —CX 2 

X3 := 4X3(1 - Xs) X 2 := -CX 2 X 2 < 0 X 3 := 4 X 3(1 - X 3 ) 

Xz := 4 x 3(1 — X 3)®3 > 0.5 



Fig. 3. A hybrid automaton that accepts Zeno executions that do not periodi- 
cally jump between the discrete states. 



Definition 12 (Zeno State). The eo limit point of a Zeno execution is called 
the Zeno state. 

We use Zoo C Q x X to denote the set of Zeno states, so that Zoo is the w limit 
set of the Zeno execution. We write Qoo for the discrete part of Zoo and Eao for 
the corresponding edges. In Example 1, we have 

^00 = {{qi, (0, 0)), (92, (3, 0)), (52, (5, 0))}, 

Qoo = {51, 52}, and Eoo = E. 

It is easy to construct an example with a Zeno executions that do not have 
a Zeno state. The idea is to let the continuous part of the execution become 
unbounded as t ^ 'Too(x). It is also straightforward to derive examples where 
the set of Zeno states have any number of elements, as well as an infinite but 
countable or uncountable number of elements. An interesting question is if for a 
Zeno execution y = (5, x, r), the discrete part q must become periodic for t G t 
sufficiently close to Too(x), as in Example 1. The answer is no as illustrated by 
the following example. 

Example 2. Consider the Zeno hybrid automaton in Figure 3 (cf. Example 1). 
This system does not accept Zeno executions that periodically jump between 
the two discrete states. A simulation is presented in Figure 4, where Xi and X 2 
are shown. The third continuous state is initialized at X 3 {tq) = 0.9. The reason 
for the quasi-periodic behavior is that the reset map of X3 is the logistic map 
and iteration of this map will give any value in (0, 1), e.g., [10]. 

A reset relation Reset is non-expanding, if there exists S G [0, 1] such that 
{q',x') G Reset(g, x) implies ||x'|| < (5||x||. It is contracting, if there exists S G 
[0, 1) such that {q',x') G Reset(g, x) and {q' ,y') G Reset(g, y) imply jjx' — y'jj < 
i5||y — xjj. Note that the reset relation has to be a function in the second case. 
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Fig. 4. An example of an execution for the hybrid automaton in Example 2. The 
continuous states x\ (solid) and (dotted) are shown. Note how they illustrate 
the quasi-periodicity. 



For smooth dynamical systems, a Lipschitz assumption on the vector field 
excludes finite escape time. This is not a sufficient condition for hybrid systems. 
However, if the reset relation is non-expanding (in addition to the Lipschitz 
assumption on /(<?,•))) then the continuous state is bounded along executions. 

Lemma 2. Consider a hybrid automaton with non-expanding reset relation. 
Then, there exists c > 0 such that for all executions x = {t, <7, x) € Eh and 
ter, 

||x(t)||<(||x(ro)|| + l)e^(‘-^o)-l. 

Proof. The proof, see [15], is similar to the corresponding result for continuous 
systems [10, Proposition 5.3]. 

When x{-) is bounded, the Bolzano- Weierstrass Property implies that there 
exists at least one Zeno state for each Zeno execution. If the continuous part of 
the reset relation is the identity map, then the continuous part of the Zeno state 
is a singleton, as proved next. 

Theorem 2. Consider a hybrid automaton such that {q',x') G Heset{q,x) im- 
plies x' = X. Then, for every Zeno execution x = {x,q,x), it holds that Z^o = 
Qoo X {x} for some Qoo C Q and x G'X.. 

Proof. For all sequences ^ ^ suppose 6i G 

[xrii ) TraJ) where rii ^ oo as i ^ oo. We have 



x{9i) 



x{Tm)+ / f{q{Tm),x{T)) dr 

x{Tm) + {Oi - r„J/(g(r„J, (a;i(^ij, • ■ 
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for some . . . , G [Tni,T^.]. Hence, for all k > £ > 0, 
x{0k) = x{0e) + - 0e)f{q{Tnt), ■ ■ ■ > 

nfc — 1 

+ {0 k- Tn^ )/ (g(r„, ),{xi{^nJ,---,Xn{^njf), 



which gives that 



rife 

||a:(6'fe) - x{0e)\\ < K {t[ - n), 

i—Ui 



where K > 0 is a constant such that \\f{q,x)\\ < K for all {q,x) G Q x X. 
Such constant exists due to Lemma 2. By the fact that ~ ^ 

we know that {x{0i)}'l^Q is a Cauchy sequence. The space X = R” is complete, 
so the sequence has a limit x = limi^oo x{0i). This limit is independent of the 
choice of sequence follows from the following argument. Consider two 



sequences {ai}to G x, such that a 


i Too and Pi Too- 


Suppose G [Tmi,T{ni] and f3i G where mt 

i oo, and rrii > Ui. Then, 


oo and rii ^ oo as 


x{ai) = x{/3i) + (r^. - Pi)f{q{Tm), {xi{fY’ ■ ■ 


■,Xn{fZ)r) 


rrii — l 


■,xn{f-)f) 


j=rii + l 




+ (a* - Tmi)f{q{Tmi), {xi{£.l„J, ■■■, 


Xniiljf). 



This gives that \\x{ai) - x{l3i)\\ < X (t' - r^). Hence, \\x{ai) - x{!3i)\\ ^0 

as t ^ oo, which shows that both sequences have the same limit. This completes 
the proof. 



Note that Theorem 2 gives the structure of the Zeno state for the large class of 
hybrid systems called switched systems [9], since these systems can be modeled 
as hybrid automata with identity reset relation. 

If the reset relation is contracting and (g', x') G Reset(( 7 , 0) implies that x' is 
the origin, then the continuous part of the Zeno state is also the origin. 

Theorem 3. Consider a Zeno hybrid automaton with contracting reset relation 
and such that {q',x') G Reset(g, 0) implies x' = 0. Then, for every Zeno execu- 
tion X = {x,q,x), it holds that Z^o = Qoo x {0} for some Qoo ^ Q- 

Proof. For all sequences {0i}^Q, 0i G r, such that 0i Too, suppose 0i G 
[r„^,T^.], where ^ oo as i ^ oo. We have 



||a:(6»*)|| < ||a;(r„J|| + 



f{q{xrH),x{T)) dr 



< l|a;(r„J|| +K{t' -t„J, 
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where > 0 is the same constant as in the proof of Theorem 2. Using the fact 
that ||a;(r„J|| < <5||a;(T^._;^)||, it follows that 



|a;( 6 »i)|| < < 5 ||a;(T' _i)|| + K{t' - 



= (5 



x(r„,_i) + 



f{q{Trn-i),x{T)) dr 



+ K{t' -r„J 



< < 5 ||a;(T„,_i)|| + KS{t^ I - r„,_i) + K{t' - 



By induction, 

rii 

||a^(0.)|| < ^”ia:(ro)|| + K ~ r™). 

m—0 



Since 



oo rii oo oo 

Y^KY, -Tm) = KY (4 - Tm) E 

ni=0 m—O m=0 ni—O 

it holds that X)m=o ~ '^m) ^ 0 as Uj ^ oo. This yields that 
||a;(0i)|i ^ 0 as i ^ oo, which, hence, completes the proof. 

A generalization of Theorem 3 holds if we change the assumption to that {q' , x') G 
Reset(g, X*) implies x' = x* for some x* G Dom, see [15]. 

For a large class of Zeno hybrid automata, the continuous part of the Zeno 
state is located on the intersection of the boundaries of Dom(( 7 , •) for q G Qoo- 
Next this result is stated for hybrid automata with non-expanding reset relation. 
Recall that I{q) = {x G X : (q,x) G Dom}. 

Proposition 7. Consider a hybrid automaton H with non-expanding reset re- 
lation. Assume it accepts a Zeno execution x = (u q, x) G with set of Zeno 
states Zoo = {(9i) N > 1. If, for all i G {1,...,X| and x G I{qi)°, 
Reset(gi,a;) = 0, then Xi G dl{qi) for all i G {l,...,Xj. Furthermore, if there 
exists X G Dom such that for all i G {1, . ■ ■ , N}, Xi = x, then x G Hfci dl{qi). 

Proof. See [15]. 

It follows from Proposition 7 that if the boundaries of /(•) are not intersecting, 
then there exist no Zeno executions with non-empty Zeno state and fV > 1. 
Proposition 7 is thus a refinement of the condition given in Proposition 5, which 
states that a hybrid automaton is non-Zeno if the graph (Q, E) has no cycle. 

5 Conclusions 

Motivated by numerous assumptions like “In this paper, we assume that the 
switched system is live and nonZeno” [3] and suggestions like “Additional work 
is needed in determining the role that Zeno-type control might play in hybrid 




464 J. Zhang et al. 



system supervision” [6], we have extended some classical results to hybrid sys- 
tems, using tools that capture both non-Zeno and Zeno executions. We have also 
tried to illustrate some of the nature of Zeno by characterizing Zeno executions 
and Zeno states for a few quite broad classes of hybrid systems. Zeno hybrid 
automata are characterized from a geometric point of view in [11]. 
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